Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Behind the curtain - How Django handles a request

636 views

Published on

Does Django sometimes feel like Magic to you? In this presentation, we will peek behind the curtain to understand the wizardry that turns an HTTP request into a response.

Despite rumors to the contrary, Django does not actually involve any magic - it's just code! We will dive deep into this code and trace the exact steps that turn an HTTP request into a response.

On this exciting journey, you'll get a high-level overview of the different components that make up Django and dissect elegant code constructs.

Originally presented at DjangoCon Europe 2018 in Heidelberg, Germany.

Published in: Software
  • Login to see the comments

Behind the curtain - How Django handles a request

  1. 1. Behind the curtain How Django handles a request Daniel Hepper
  2. 2. About this tutorial • not a top-down Django tutorial • for people who want to get to the bottom of things
  3. 3. What we'll cover • what is Django? • dissecting the tiniest Django web app • the same app, but more complicated
  4. 4. About me Daniel Hepper • degree in computer science • freelancing for 7 years • first contact with Django around 0.95 • NOT a core developer • varying amounts beard
  5. 5. What is Django
  6. 6. The web framework for perfectionists with deadlines Django makes it easier to build better Web apps more quickly and with less code.
  7. 7. Web apps • web app • Django project • django-admin.py startproject • Django app • module, part of an web app • might be re-usable: re-usable app • python manage.py startapp
  8. 8. Web Framework
  9. 9. Web Framework • you take things from a library,
 you put something in a framework • "Don't call us, we call you" • But you have to tell us your number
 you have to provide an entrypoint
  10. 10. Settings Django's entrypoint
  11. 11. Web Framework
  12. 12. Web Framework • turn HTTP requests into HTTP response
  13. 13. Browser Cloud HTTP Request HTTP Response
  14. 14. HTTP Client HTTP Request HTTP Response HTTP Server
  15. 15. HTTP-Request & Reponse GET / HTTP/1.1 Host: 127.0.0.1:8000 User-Agent: curl/7.54.0 Accept: */* HTTP/1.0 200 OK Date: Sat, 26 May 2018 07:27:52 GMT Server: WSGIServer/0.2 CPython/3.6.4 Content-Type: text/html; charset=utf-8 Hello world
  16. 16. HTTP Client HTTP HTTP Server Django ?
  17. 17. HTTP Client HTTP HTTP Server Django WSGI
  18. 18. Browser HTTP Gunicorn Django WSGI
  19. 19. Browser HTTP Nginx Django WSGI Gunicorn HTTP
  20. 20. Browser HTTP Nginx Django WSGI uwsgi uwsgi
  21. 21. WSGI • Defined in PEP-333 / PEP-3333 • "A [...] standard interface between web servers and Python web applications or frameworks, to promote web application portability across a variety of web servers." • "WSGI is a tool for framework and server developers"
  22. 22. WSGI • servers • applications • middleware
  23. 23. WSGI Application • Callable that accepts two arguments • a function, method, class, instance with a __call__ method • result = application(environ, start_response) • environ: dictionary object, containing CGI-style environment variables • start_response: callable, accepting two required positional arguments, third optional • def start_response(status, response_headers, exc_info=None): • When called by the server, the application object must return an iterable yielding zero or more bytestrings
  24. 24. Hello WSGI HELLO_WORLD = b"Hello world!n" def simple_app(environ, start_response): """Simplest possible application object""" status = '200 OK' response_headers = [('Content-type', 'text/plain')] start_response(status, response_headers) return [HELLO_WORLD]
  25. 25. The web framework for perfectionists with deadlines Django makes it easier to build better Web apps more quickly and with less code. WSGI
  26. 26. Let's create a Django project
  27. 27. (venv)$ pip install django==2.0.5 (venv)$ django-admin.py startproject default (venv)$ tree default default ├── manage.py └── default ├── __init__.py ├── settings.py ├── urls.py └── wsgi.py (venv)$ cat default/default/settings.py
  28. 28. """ Django settings for mysite project... """ import os # Build paths inside the project like this: os.path.join(BASE_DIR, ...) BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) # Quick-start development settings - unsuitable for production # See https://docs.djangoproject.com/en/2.0/howto/deployment/checklist/ # SECURITY WARNING: keep the secret key used in production secret! SECRET_KEY = '-^oi_8%de()$d3qqa_4fgwds#&nlfn2(j-@_hu(j-m7-+-@q@p' # SECURITY WARNING: don't run with debug turned on in production! DEBUG = True ALLOWED_HOSTS = [] # Application definition INSTALLED_APPS = [ 'django.contrib.admin', ... 'django.contrib.staticfiles', ] MIDDLEWARE = [ 'django.middleware.security.SecurityMiddleware', ... 'django.middleware.clickjacking.XFrameOptionsMiddleware', ] ROOT_URLCONF = 'default.urls' TEMPLATES = [ { 'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [], 'APP_DIRS': True, 'OPTIONS': { 'context_processors': [ 'django.template.context_processors.debug', "Django is too complicated!"
  29. 29. Web development is complicated Django offers reasonable defaults to make your life easier
  30. 30. Let's simplify
  31. 31. All you need is... • a view • an URLConf • settings • a WSGI application object
  32. 32. https://github.com/consideratecode/django-behind-the-curtain/
  33. 33. views.py from django.http import HttpResponse def hello(request): return HttpResponse('Hello world')
  34. 34. urls.py from django.urls import path from . import views urlpatterns = [ path('', views) ]
  35. 35. settings.py SECRET_KEY = 'not-so-secret-123%^&' ROOT_URLCONF = 'simple.urls'
  36. 36. wsgi.py import os from django.core.wsgi import get_wsgi_application os.environ.setdefault( "DJANGO_SETTINGS_MODULE", "simple.settings" ) application = get_wsgi_application()
  37. 37. Running a Django web application • job of a WSGI server • two phases • import the application object • call the application object for each request
  38. 38. A simple WSGI server from wsgiref.simple_server import make_server from simple.wsgi import application httpd = make_server('127.0.0.1', 8000, application) print("Serving HTTP on port 8000...") httpd.handle_request()
  39. 39. (venv)$ tree simple . ├── server.py └── simple ├── __init__.py ├── __pycache__ ├── settings.py ├── urls.py ├── views.py └── wsgi.py
  40. 40. DEMO
  41. 41. Behind the curtain
  42. 42. Starting the application
  43. 43. Starting the application
  44. 44. import os from django.core.wsgi import get_wsgi_application os.environ.setdefault( "DJANGO_SETTINGS_MODULE", "simple.settings" ) application = get_wsgi_application() wsgi.py
  45. 45. import django from django.core.handlers.wsgi import WSGIHandler def get_wsgi_application(): """ The public interface to Django's WSGI support. Return a WSGI callable. Avoids making django.core.handlers.WSGIHandler a public API, in case the internal WSGI implementation changes or moves in the future. """ django.setup(set_prefix=False) return WSGIHandler() django/core/wsgi.py
  46. 46. from django.utils.version import get_version VERSION = (2, 0, 0, 'final', 0) __version__ = get_version(VERSION) def setup(set_prefix=True): """ Configure the settings (this happens as a side effect of accessing the first setting), configure logging and populate the app registry. Set the thread-local urlresolvers script prefix if `set_prefix` is True. """ from django.apps import apps from django.conf import settings from django.urls import set_script_prefix from django.utils.log import configure_logging configure_logging(settings.LOGGING_CONFIG, settings.LOGGING) if set_prefix: set_script_prefix( '/' if settings.FORCE_SCRIPT_NAME is None else settings.FORCE_SCRIPT_NAME ) apps.populate(settings.INSTALLED_APPS) django/__init__.py
  47. 47. [...] class LazySettings(LazyObject): """ A lazy proxy for either global Django settings or a custom settings object. The user can manually configure settings prior to using them. Otherwise,Django uses the settings module pointed to by DJANGO_SETTINGS_MODULE. """ def __getattr__(self, name): """ Return the value of a setting and cache it in self.__dict__. """ if self._wrapped is empty: self._setup(name) val = getattr(self._wrapped, name) self.__dict__[name] = val return val [...] settings = LazySettings() django/conf/__init__.py
  48. 48. [...] class LazySettings(LazyObject): """ A lazy proxy for either global Django settings or a custom settings object. The user can manually configure settings prior to using them. Otherwise,Django uses the settings module pointed to by DJANGO_SETTINGS_MODULE. """ def _setup(self, name=None): """ Load the settings module pointed to by the environment variable. This is used the first time settings are needed, if the user hasn't configured settings manually. """ settings_module = os.environ.get(ENVIRONMENT_VARIABLE) if not settings_module: desc = ("setting %s" % name) if name else "settings" raise ImproperlyConfigured(...) self._wrapped = Settings(settings_module) [...] settings = LazySettings() django/conf/__init__.py
  49. 49. [...] class Settings: def __init__(self, settings_module): for setting in dir(global_settings): if setting.isupper(): setattr(self, setting, getattr(global_settings, setting)) [...] mod = importlib.import_module(self.SETTINGS_MODULE) self._explicit_settings = set() for setting in dir(mod): if setting.isupper(): setting_value = getattr(mod, setting) [...] setattr(self, setting, setting_value) [...] [...] settings = LazySettings() django/conf/__init__.py
  50. 50. from django.utils.version import get_version VERSION = (2, 0, 0, 'final', 0) __version__ = get_version(VERSION) def setup(set_prefix=True): """ Configure the settings (this happens as a side effect of accessing the first setting), configure logging and populate the app registry. Set the thread-local urlresolvers script prefix if `set_prefix` is True. """ from django.apps import apps from django.conf import settings from django.urls import set_script_prefix from django.utils.log import configure_logging configure_logging(settings.LOGGING_CONFIG, settings.LOGGING) if set_prefix: set_script_prefix( '/' if settings.FORCE_SCRIPT_NAME is None else settings.FORCE_SCRIPT_NAME ) apps.populate(settings.INSTALLED_APPS) django/__init__.py
  51. 51. import django from django.core.handlers.wsgi import WSGIHandler def get_wsgi_application(): """ The public interface to Django's WSGI support. Return a WSGI callable. Avoids making django.core.handlers.WSGIHandler a public API, in case the internal WSGI implementation changes or moves in the future. """ django.setup(set_prefix=False) return WSGIHandler() django/core/wsgi.py
  52. 52. class WSGIHandler(base.BaseHandler): request_class = WSGIRequest def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self.load_middleware() def __call__(self, environ, start_response): [...] django/core/handlers/wsgi.py
  53. 53. class BaseHandler: [...] def load_middleware(self): """ Populate middleware lists from settings.MIDDLEWARE. """ self._request_middleware = [] self._view_middleware = [] self._template_response_middleware = [] self._response_middleware = [] self._exception_middleware = [] handler = convert_exception_to_response( self._get_response) for middleware_path in reversed(settings.MIDDLEWARE): [...] self._middleware_chain = handler django/core/handlers/base.py
  54. 54. class WSGIHandler(base.BaseHandler): [...] def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self.load_middleware() def __call__(self, environ, start_response): [...] django/core/handlers/wsgi.py
  55. 55. import django from django.core.handlers.wsgi import WSGIHandler def get_wsgi_application(): """ The public interface to Django's WSGI support. Return a WSGI callable. Avoids making django.core.handlers.WSGIHandler a public API, in case the internal WSGI implementation changes or moves in the future. """ django.setup(set_prefix=False) return WSGIHandler() django/core/wsgi.py
  56. 56. import os from django.core.wsgi import get_wsgi_application os.environ.setdefault( "DJANGO_SETTINGS_MODULE", "simple.settings" ) application = get_wsgi_application() wsgi.py
  57. 57. Handling a request
  58. 58. WSGI Application • Callable that accepts two arguments • a function, method, class, instance with a __call__ method • result = application(environ, start_response) • environ: dictionary object, containing CGI-style environment variables • start_response: callable, accepting two required positional arguments, third optional • def start_response(status, response_headers, exc_info=None): • When called by the server, the application object must return an iterable yielding zero or more bytestrings
  59. 59. class WSGIHandler(base.BaseHandler): request_class = WSGIRequest [...] def __call__(self, environ, start_response): set_script_prefix(get_script_name(environ)) signals.request_started.send(sender=self.__class__, environ=environ) request = self.request_class(environ) response = self.get_response(request) [...] django/core/handlers/wsgi.py
  60. 60. class WSGIRequest(HttpRequest): def __init__(self, environ): [...] django/core/handlers/wsgi.py
  61. 61. class WSGIRequest(HttpRequest): def __init__(self, environ): [...] django/core/handlers/wsgi.py
  62. 62. class HttpRequest: """A basic HTTP request.""" # The encoding used in GET/POST dicts. None means use default # setting. _encoding = None _upload_handlers = [] def __init__(self): # WARNING: The `WSGIRequest` subclass doesn't call # `super`. # Any variable assignment made here should also happen in # `WSGIRequest.__init__()`. [...] django/core/http/request.py
  63. 63. class WSGIRequest(HttpRequest): def __init__(self, environ): script_name = get_script_name(environ) path_info = get_path_info(environ) if not path_info: # Sometimes PATH_INFO exists, but is empty (e.g. # accessing the SCRIPT_NAME URL without a # slash). We really need to operate as if they'd # requested '/'. Not amazingly nice to force # the path like this, but should be harmless. path_info = '/' [...] django/core/handlers/wsgi.py
  64. 64. path vs. script_name & path_info • http://mydjangoproject.com/ • http://mydjangoproject.com/polls/ • http://mydjangoprojects.com/project1/ • http://mydjangoprojects.com/project2/polls/ • http://mydjangoprojects.com/project3
  65. 65. class WSGIRequest(HttpRequest): def __init__(self, environ): script_name = get_script_name(environ) path_info = get_path_info(environ) if not path_info: # Sometimes PATH_INFO exists, but is empty (e.g. # accessing the SCRIPT_NAME URL without a # slash). We really need to operate as if they'd # requested '/'. Not amazingly nice to force # the path like this, but should be harmless. path_info = '/' [...] def get_script_name(environ): """ Return the equivalent of the HTTP request's SCRIPT_NAME environment variable. If Apache mod_rewrite is used, return what would have been the script name prior to any rewriting (so it's the script name as seen from the client's perspective), unless the FORCE_SCRIPT_NAME setting is set (to anything). """ [...] django/core/handlers/wsgi.py
  66. 66. class WSGIRequest(HttpRequest): def __init__(self, environ): script_name = get_script_name(environ) path_info = get_path_info(environ) if not path_info: # Sometimes PATH_INFO exists, but is empty (e.g. # accessing the SCRIPT_NAME URL without a # slash). We really need to operate as if they'd # requested '/'. Not amazingly nice to force # the path like this, but should be harmless. path_info = '/' [...] def get_path_info(environ): """Return the HTTP request's PATH_INFO as a string.""" path_info = get_bytes_from_wsgi(environ, 'PATH_INFO', '/') return repercent_broken_unicode(path_info).decode() django/core/handlers/wsgi.py
  67. 67. class WSGIRequest(HttpRequest): def __init__(self, environ): [...] self.environ = environ self.path_info = path_info # be careful to only replace the first slash in the path # because of http://test/something and # http://test//something being different as # stated in http://www.ietf.org/rfc/rfc2396.txt self.path = '%s/%s' % (script_name.rstrip('/'), path_info.replace('/', '', 1)) [...] django/core/handlers/wsgi.py
  68. 68. class WSGIRequest(HttpRequest): def __init__(self, environ): [...] self.META = environ self.META['PATH_INFO'] = path_info self.META['SCRIPT_NAME'] = script_name self.method = environ['REQUEST_METHOD'].upper() [...] django/core/handlers/wsgi.py
  69. 69. class WSGIRequest(HttpRequest): def __init__(self, environ): [...] self.content_type, self.content_params = cgi.parse_header(environ.get('CONTENT_TYPE', '')) if 'charset' in self.content_params: try: codecs.lookup(self.content_params['charset']) except LookupError: pass else: self.encoding = self.content_params['charset'] [...] django/core/handlers/wsgi.py
  70. 70. class WSGIRequest(HttpRequest): def __init__(self, environ): [...] self._post_parse_error = False try: content_length = int(environ.get('CONTENT_LENGTH')) except (ValueError, TypeError): content_length = 0 self._stream = LimitedStream(self.environ['wsgi.input'], content_length) self._read_started = False self.resolver_match = None django/core/handlers/wsgi.py
  71. 71. class WSGIRequest(HttpRequest): def __init__(self, environ): [...] self._post_parse_error = False try: content_length = int(environ.get('CONTENT_LENGTH')) except (ValueError, TypeError): content_length = 0 self._stream = LimitedStream(self.environ['wsgi.input'], content_length) self._read_started = False self.resolver_match = None class LimitedStream: """ Wrap another stream to disallow reading it past a number of bytes. """ [...] django/core/handlers/wsgi.py
  72. 72. class WSGIHandler(base.BaseHandler): request_class = WSGIRequest [...] def __call__(self, environ, start_response): set_script_prefix(get_script_name(environ)) signals.request_started.send(sender=self.__class__, environ=environ) request = self.request_class(environ) response = self.get_response(request) [...] django/core/handlers/wsgi.py
  73. 73. class BaseHandler: [...] def get_response(self, request): """ Return an HttpResponse object for the given HttpRequest. """ # Setup default url resolver for this thread set_urlconf(settings.ROOT_URLCONF) response = self._middleware_chain(request) django/core/handlers/base.py
  74. 74. # Overridden URLconfs for each thread are stored here. _urlconfs = local() [...] def set_urlconf(urlconf_name): """ Set the URLconf for the current thread (overriding the default one in settings). If urlconf_name is None, revert back to the default. """ if urlconf_name: _urlconfs.value = urlconf_name else: if hasattr(_urlconfs, "value"): del _urlconfs.value django/urls/base.py
  75. 75. class BaseHandler: [...] def get_response(self, request): """ Return an HttpResponse object for the given HttpRequest. """ # Setup default url resolver for this thread set_urlconf(settings.ROOT_URLCONF) response = self._middleware_chain(request) django/core/handlers/base.py
  76. 76. class BaseHandler: def load_middleware(self): """ Populate middleware lists from settings.MIDDLEWARE. """ handler = convert_exception_to_response( self._get_response ) [...] self._middleware_chain = handler def get_response(self, request): """ Return an HttpResponse object for the given HttpRequest. """ # Setup default url resolver for this thread set_urlconf(settings.ROOT_URLCONF) response = self._middleware_chain(request) django/core/handlers/base.py
  77. 77. class BaseHandler: [...] def _get_response(self, request): """ Resolve and call the view, then apply view, exception, and template_response middleware. This method is everything that happens inside the request/response middleware. """ [...] django/core/handlers/base.py
  78. 78. class BaseHandler: [...] def _get_response(self, request): """ Resolve and call the view, then apply view, exception, and template_response middleware. This method is everything that happens inside the request/response middleware. """ response = None if hasattr(request, 'urlconf'): urlconf = request.urlconf set_urlconf(urlconf) resolver = get_resolver(urlconf) else: resolver = get_resolver() resolver_match = resolver.resolve(request.path_info) callback, callback_args, callback_kwargs = resolver_match request.resolver_match = resolver_match django/core/handlers/base.py
  79. 79. class BaseHandler: [...] def _get_response(self, request): [...] callback, callback_args, callback_kwargs = resolver_match [...] # Apply view middleware for middleware_method in self._view_middleware: response = middleware_method(request, callback, ...) if response: break [...] django/core/handlers/base.py
  80. 80. class BaseHandler: [...] def _get_response(self, request): [...] callback, callback_args, callback_kwargs = resolver_match [...] # Apply view middleware [...] if response is None: wrapped_callback = self.make_view_atomic(callback) try: response = wrapped_callback( request, *callback_args, **callback_kwargs ) except Exception as e: response = self.process_exception_by_middleware( e, request ) django/core/handlers/base.py
  81. 81. class BaseHandler: [...] def _get_response(self, request): [...] callback, callback_args, callback_kwargs = resolver_match [...] # Apply view middleware [...] response = wrapped_callback( request, *callback_args, **callback_kwargs ) # Complain if the view returned None (a common error). [...] # If the response supports deferred rendering, apply # template response middleware and then render the response [...] return response django/core/handlers/base.py
  82. 82. class BaseHandler: def load_middleware(self): """ Populate middleware lists from settings.MIDDLEWARE. """ handler = convert_exception_to_response( self._get_response ) [...] self._middleware_chain = handler def get_response(self, request): """ Return an HttpResponse object for the given HttpRequest. """ # Setup default url resolver for this thread set_urlconf(settings.ROOT_URLCONF) response = self._middleware_chain(request) django/core/handlers/base.py
  83. 83. class BaseHandler: [...] def get_response(self, request): [...] response = self._middleware_chain(request) response._closable_objects.append(request) [...] return response django/core/handlers/base.py
  84. 84. class HttpResponseBase: [...] # The WSGI server must call this method upon completion of # the request. # See http://blog.dscpl.com.au/2012/10/obligations-for-calling-close def close(self): for closable in self._closable_objects: try: closable.close() except Exception: pass self.closed = True signals.request_finished.send(sender=self._handler_class) django/http/response.py
  85. 85. class BaseHandler: [...] def get_response(self, request): [...] response = self._middleware_chain(request) response._closable_objects.append(request) # If the exception handler returns a TemplateResponse # that has not been rendered, force it to be rendered. if not getattr(response, 'is_rendered', True) and callable(getattr(response, 'render', None)): response = response.render() [...] return response django/core/handlers/base.py
  86. 86. class BaseHandler: [...] def get_response(self, request): [...] response = self._middleware_chain(request) response._closable_objects.append(request) # If the exception handler returns a TemplateResponse # that has not been rendered, force it to be rendered. if not getattr(response, 'is_rendered', True) and callable(getattr(response, 'render', None)): response = response.render() if response.status_code == 404: logger.warning( 'Not Found: %s', request.path, extra={'status_code': 404, 'request': request}) return response django/core/handlers/base.py
  87. 87. class WSGIHandler(base.BaseHandler): request_class = WSGIRequest [...] def __call__(self, environ, start_response): set_script_prefix(get_script_name(environ)) signals.request_started.send(sender=self.__class__, environ=environ) request = self.request_class(environ) response = self.get_response(request) [...] django/core/handlers/wsgi.py
  88. 88. class WSGIHandler(base.BaseHandler): request_class = WSGIRequest [...] def __call__(self, environ, start_response): [...] response = self.get_response(request) response._handler_class = self.__class__ status = '%d %s' % (response.status_code, response.reason_phrase response_headers = list(response.items()) for c in response.cookies.values(): response_headers.append(('Set-Cookie', c.output(header=''))) start_response(status, response_headers) if (getattr(response, 'file_to_stream', None) is not None and environ.get('wsgi.file_wrapper')): response = environ['wsgi.file_wrapper'](response.file_to_str return response django/core/handlers/wsgi.py
  89. 89. Middleware
  90. 90. Middleware power Django
  91. 91. Default MIDDLEWARE = [ 'django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', ]
  92. 92. Function based middleware def simple_middleware(get_response): # One-time configuration and initialization. def middleware(request): # Code to be executed for each request before # the view (and later middleware) are called. response = get_response(request) # Code to be executed for each request/response after # the view is called. return response return middleware
  93. 93. Class based middleware class SimpleMiddleware: def __init__(self, get_response): self.get_response = get_response # One-time configuration and initialization. def __call__(self, request): # Code to be executed for each request before # the view (and later middleware) are called. response = self.get_response(request) # Code to be executed for each request/response after # the view is called. return response
  94. 94. Additional hooks • process_view(request, view_func, view_args, view_kwargs) • process_exception(request, exception) • process_template_response(request, response)
  95. 95. Pre-Django 1.10-style middleware • process_request() • process_response() • django.utils.deprecation.MiddlewareMixin
  96. 96. Using middleware
  97. 97. class Middleware1: def __init__(self, get_response): print('> Middleware1.__init__') self.get_response = get_response def __call__(self, request): print('> Middleware1.__call__') response = self.get_response(request) print('< Middleware1.__call__') return response def process_view(self, request, view_func, view_args, view_kwargs): print('> Middleware1.process_view') return None def process_exception(self, request, exception): print('> Middleware1.process_exception') return None def process_template_response(self, request, response): print('> Middleware1.process_template_response') return response middleware/middleware.py
  98. 98. SECRET_KEY = 'not-so-secret-123%^&' ROOT_URLCONF = 'middleware.urls' MIDDLEWARE = [ 'middleware.middleware.Middleware1', 'middleware.middleware.Middleware2', 'middleware.middleware.Middleware3', ] middleware/settings.py
  99. 99. Starting the application
  100. 100. $ python server.py > Middleware3.__init__ > Middleware2.__init__ > Middleware1.__init__ Serving HTTP on port 8000...
  101. 101. class BaseHandler: [...] def load_middleware(self): """ Populate middleware lists from settings.MIDDLEWARE. """ self._request_middleware = [] self._view_middleware = [] self._template_response_middleware = [] self._response_middleware = [] self._exception_middleware = [] handler = convert_exception_to_response(self._get_response) for middleware_path in reversed(settings.MIDDLEWARE): [...] self._middleware_chain = handler django/core/handlers/base.py
  102. 102. class BaseHandler: [...] def load_middleware(self): """ Populate middleware lists from settings.MIDDLEWARE. """ [...] handler = convert_exception_to_response(self._get_response) for middleware_path in reversed(settings.MIDDLEWARE): middleware = import_string(middleware_path) try: mw_instance = middleware(handler) except MiddlewareNotUsed as exc: if settings.DEBUG: [...] continue [...] handler = convert_exception_to_response(mw_instance) self._middleware_chain = handler django/core/handlers/base.py
  103. 103. settings.MIDDLEWARE = [ 'middleware.middleware.Middleware1', 'middleware.middleware.Middleware2', 'middleware.middleware.Middleware3', ] handler = convert_exception_to_response(self._get_response) for middleware_path in reversed(settings.MIDDLEWARE): middleware = import_string(middleware_path) mw_instance = middleware(handler) handler = convert_exception_to_response(mw_instance) handler = convert_exception_to_response(self._get_response) handler = convert_exception_to_response(Middleware3(handler)) handler = convert_exception_to_response(Middleware2(handler)) handler = convert_exception_to_response(Middleware1(handler))
  104. 104. settings.MIDDLEWARE = [ 'middleware1', 'middleware2', 'middleware3', ] handler = convert_exception_to_response(self._get_response) for middleware_path in reversed(settings.MIDDLEWARE): middleware = import_string(middleware_path) mw_instance = middleware(handler) handler = convert_exception_to_response(mw_instance) handler = middleware1(middleware2(middleware3(self._get_response)))))))
  105. 105. class BaseHandler: [...] def load_middleware(self): """ Populate middleware lists from settings.MIDDLEWARE. """ [...] handler = convert_exception_to_response(self._get_response) for middleware_path in reversed(settings.MIDDLEWARE): middleware = import_string(middleware_path) try: mw_instance = middleware(handler) except MiddlewareNotUsed as exc: if settings.DEBUG: [...] continue [...] handler = convert_exception_to_response(mw_instance) self._middleware_chain = handler django/core/handlers/base.py
  106. 106. class BaseHandler: [...] def load_middleware(self): [...] for middleware_path in reversed(settings.MIDDLEWARE): [...] if hasattr(mw_instance, 'process_view'): self._view_middleware.insert( 0, mw_instance.process_view) if hasattr(mw_instance, 'process_template_response'): self._template_response_middleware.append( mw_instance.process_template_response) if hasattr(mw_instance, 'process_exception'): self._exception_middleware.append( mw_instance.process_exception) handler = convert_exception_to_response(mw_instance) self._middleware_chain = handler django/core/handlers/base.py
  107. 107. settings.MIDDLEWARE = [ 'middleware1', 'middleware2', 'middleware3', ] self._view_middleware = [ middleware1_instance.process_view, middleware2_instance.process_view, middleware3_instance.process_view, ] self._template_response_middleware = [ middleware3_instance.process_template_response, middleware2_instance.process_template_response, middleware1_instance.process_template_response, ] self._exception_middleware = [ middleware3_instance.process_exception, middleware2_instance.process_exception, middleware1_instance.process_exception, ]
  108. 108. • DIAGRAM
  109. 109. Handling a request
  110. 110. $ python server.py > Middleware3.__init__ > Middleware2.__init__ > Middleware1.__init__ Serving HTTP on port 8000... > Middleware1.__call__ > Middleware2.__call__ > Middleware3.__call__ > Middleware1.process_view > Middleware2.process_view > Middleware3.process_view < Middleware3.__call__ < Middleware2.__call__ < Middleware1.__call__
  111. 111. class BaseHandler: def load_middleware(self): """ Populate middleware lists from settings.MIDDLEWARE. """ [...] def get_response(self, request): """ Return an HttpResponse object for the given HttpRequest. """ # Setup default url resolver for this thread set_urlconf(settings.ROOT_URLCONF) response = self._middleware_chain(request) middleware1(middleware2(middleware3(self._get_response))))))) django/core/handlers/base.py
  112. 112. Middleware1 Middleware2 __call__ Middleware3 __call__ BaseHandler _get_response
  113. 113. class BaseHandler: [...] def _get_response(self, request): """ Resolve and call the view, then apply view, exception, and template_response middleware. This method is everything that happen inside the request/response middleware. """ response = None [...] return response django/core/handlers/base.py
  114. 114. class BaseHandler: [...] def _get_response(self, request): """ Resolve and call the view, then apply view, exception, and template_response middleware. This method is everything that happen inside the request/response middleware. """ response = None if hasattr(request, 'urlconf'): urlconf = request.urlconf set_urlconf(urlconf) resolver = get_resolver(urlconf) else: resolver = get_resolver() [...] return response django/core/handlers/base.py
  115. 115. [...] @functools.lru_cache(maxsize=None) def get_resolver(urlconf=None): if urlconf is None: from django.conf import settings urlconf = settings.ROOT_URLCONF return URLResolver(RegexPattern(r'^/'), urlconf) [...] django/urls/resolvers/base.py
  116. 116. class BaseHandler: [...] def _get_response(self, request): """ Resolve and call the view, then apply view, exception, and template_response middleware. This method is everything that happen inside the request/response middleware. """ [...] resolver_match = resolver.resolve(request.path_info) callback, callback_args, callback_kwargs = resolver_match request.resolver_match = resolver_match [...] return response django/core/handlers/base.py
  117. 117. class BaseHandler: [...] def _get_response(self, request): """ Resolve and call the view, then apply view, exception, and template_response middleware. This method is everything that happen inside the request/response middleware. """ [...] callback, callback_args, callback_kwargs = resolver_match [...] # Apply view middleware for middleware_method in self._view_middleware: response = middleware_method(request, callback, callback_args, callback_kwargs) if response: break [...] return response django/core/handlers/base.py
  118. 118. Middleware1 Middleware2 __call__ Middleware3 __call__ BaseHandler _get_response process_view process_view process_view
  119. 119. class BaseHandler: [...] def _get_response(self, request): """ Resolve and call the view, then apply view, exception, and template_response middleware. This method is everything that happen inside the request/response middleware. """ [...] if response is None: wrapped_callback = self.make_view_atomic(callback) try: response = wrapped_callback(request, *callback_args, **callback_kwargs) except Exception as e: response = self.process_exception_by_middleware( e, request) [...] return response django/core/handlers/base.py
  120. 120. class BaseHandler: [...] def _get_response(self, request): """ Resolve and call the view, then apply view, exception, and template_response middleware. This method is everything that happen inside the request/response middleware. """ [...] # Complain if the view returned None (a common error). if response is None: if isinstance(callback, types.FunctionType): # FBV view_name = callback.__name__ else: # CBV view_name = callback.__class__.__name__ + '.__call__' raise ValueError( "The view %s.%s didn't return an HttpResponse object. It "returned None instead." % (callback.__module__, view_na ) [...] return response django/core/handlers/base.py
  121. 121. class BaseHandler: [...] def _get_response(self, request): """ Resolve and call the view, then apply view, exception, and template_response middleware. This method is everything that happen inside the request/response middleware. """ [...] # If the response supports deferred rendering, apply template # response middleware and then render the response elif hasattr(response, 'render') and callable(response.render): for middleware_method in self._template_response_middleware: response = middleware_method(request, response) # Complain if the template response middleware # returned None (a common error). if response is None: raise ValueError(...) [...] return response django/core/handlers/base.py
  122. 122. class BaseHandler: [...] def _get_response(self, request): """ Resolve and call the view, then apply view, exception, and template_response middleware. This method is everything that happen inside the request/response middleware. """ [...] # If the response supports deferred rendering, apply template # response middleware and then render the response elif hasattr(response, 'render') and callable(response.render): [...] try: response = response.render() except Exception as e: response = self.process_exception_by_middleware(e, reque return response django/core/handlers/base.py
  123. 123. class BaseHandler: [...] def process_exception_by_middleware(self, exception, request): """ Pass the exception to the exception middleware. If no middleware return a response for this exception, raise it. """ for middleware_method in self._exception_middleware: response = middleware_method(request, exception) if response: return response raise django/core/handlers/base.py
  124. 124. Middleware1 Middleware2 __call__ Middleware3 __call__ BaseHandler _get_response process_exception process_exception process_exception
  125. 125. class BaseHandler: [...] def _get_response(self, request): """ Resolve and call the view, then apply view, exception, and template_response middleware. This method is everything that happen inside the request/response middleware. """ [...] # If the response supports deferred rendering, apply template # response middleware and then render the response elif hasattr(response, 'render') and callable(response.render): [...] try: response = response.render() except Exception as e: response = self.process_exception_by_middleware(e, reque return response django/core/handlers/base.py
  126. 126. class BaseHandler: [...] def get_response(self, request): """ Return an HttpResponse object for the given HttpRequest. """ # Setup default url resolver for this thread set_urlconf(settings.ROOT_URLCONF) response = self._middleware_chain(request) [...] return response django/core/handlers/base.py
  127. 127. class BaseHandler: [...] def get_response(self, request): [...] response = self._middleware_chain(request) [...] return response django/core/handlers/base.py
  128. 128. class BaseHandler: [...] def get_response(self, request): [...] response = self._middleware_chain(request) response._closable_objects.append(request) # If the exception handler returns a TemplateResponse # that has not been rendered, force it to be rendered. if not getattr(response, 'is_rendered', True) and callable(getattr(response, 'render', None)): response = response.render() if response.status_code == 404: logger.warning( 'Not Found: %s', request.path, extra={'status_code': 404, 'request': request}) return response django/core/handlers/base.py
  129. 129. class WSGIHandler(base.BaseHandler): request_class = WSGIRequest [...] def __call__(self, environ, start_response): [...] response = self.get_response(request) response._handler_class = self.__class__ status = '%d %s' % (response.status_code, response.reason_phrase response_headers = list(response.items()) for c in response.cookies.values(): response_headers.append(('Set-Cookie', c.output(header=''))) start_response(status, response_headers) if (getattr(response, 'file_to_stream', None) is not None and environ.get('wsgi.file_wrapper')): response = environ['wsgi.file_wrapper'](response.file_to_str return response django/core/handlers/wsgi.py
  130. 130. TIL • Django is made by humans, not wizards • Just code, no magic
  131. 131. Room for contribution • Unused attributes _response_middleware and _request_middleware in BaseHandler • Wrong or misleading comments • ExceptionMiddleware does not exist
  132. 132. Thanks Daniel Hepper @danielhepper daniel@consideratecode.com consideratecode.com
  133. 133. Image credits • https://pixabay.com/en/interior-house-home- curtain-2570933/ • https://unsplash.com/photos/MEW1f-yu2KI • https://pixabay.com/en/rocket-launch-smoke-rocket-take- off-67723/ • https://pixabay.com/en/gears-cogs-machine- machinery-1236578/ • https://pixabay.com/en/balance-background-harmony- stacked-3356546/

×