@KevinVanWilder
Virtualenv• Isolated Python environment• Virtualenvwrapper  $ which python  /home/kevin/.virtualenvs/inuittest/bin/python ...
Installing libraries • Always work in a virtualenv$ pip freeze# ONE GAZILLION LIBRARIES!!!!(inuittest)$ pip freezeargparse...
introduction@KevinVanWilder
Hi!•   Architecture•   Development Practices•   The Django Ecosystem•   Infrastructure•   Tools
WTH is this thing?ARCHITECTURE OVERVIEW
Architecture                           RDBMS     Storage               ORM        Fixtures                           Model...
Model                          RDBMS     Storage              ORM        Fixtures                          ModelSignals   ...
Modelfrom django.db import modelsclass Poll(models.Model):    question = models.CharField(max_length=200)    pub_date = mo...
ModelBEGIN;CREATE TABLE "polls_poll" (    "id" serial NOT NULL PRIMARY KEY,    "question" varchar(200) NOT NULL,    "pub_d...
Request Response Cycle                                RDBMS     Storage                    ORM         Fixtures           ...
Request Response Cycle1. Determine which URLConf to use2. Load that URLConf and look for   urlpatterns variable3. Run thro...
Request Response Cycle: URLConf/polls                                           /polls/1/polls/1/results                  ...
Request Response Cycle: Viewfrom django.shortcuts import render_to_responsefrom polls.models import Polldef index(request)...
Request Response Cycle: Viewfrom django.http import Http404def detail(request, poll_id):    p = get_object_or_404(Poll, pk...
Request Response Cycle: Viewfrom   django.shortcuts import get_object_or_404, render_to_responsefrom   django.http import ...
Templating                          RDBMS     Storage              ORM        Fixtures                          ModelSigna...
Template{% if latest_poll_list %}    <ul>    {% for poll in latest_poll_list %}         <li><a href="/polls/{{ poll.id }}/...
The django developer in its natural habitatDJANGO ECOSYSTEM
Django Debug Toolbar• https://github.com/dcramer/django-debug-  toolbar• Must have for all your projects
South• Data schema migration     – Automatic migration creation     – Database independent     – Migration conflict detect...
Celery•   Asynchronous task queue•   Message Passing•   Green threads•   RabbitMQ, Redis, MongoDB, CouchDB, …
• Search for Django• Solr, ElasticSearch, Whoosh, Xapian
Sentry• getsentry.com• Exception & error aggregator• Open Source and SaaS options
Tastypie•   tastypieapi.org•   Webservice API framework•   REST•   HATEOAS Compliant•   Alternative to django-piston
Pinax•   pinaxproject.com•   Standard project layout•   Starter projects•   Default templates for prototyping•   Reusable ...
Pinax Reusable Apps•   Agon (points & positions)         •   Partitions (querysets)•   Agon-ratings                      •...
django-extensions• Shell_plus  – Autoloading of the apps database models• Additional database fields  – AutoSlugField  – C...
INFRASTRUCTURE
Application Server (AS)Web Server Gateway Interface (WSGI)• PEP 333 [P. J. Eby, dec 2010]• Common ground for portable web ...
AS: gunicorn•   “green unicorn” (cfr. green threading)•   Pre-fork worker model•   4-12 workers = 1000’s req/sec•   Each w...
AS: uWSGI• Protocol “uwsgi” (lowercase)• Included in Nginx & Cherokee• Faster than gunicorn
TOOLS
Tools• IPython   – Tab completion in the interpreter!• import ipdb; ipdb.set_trace()   – Just like pdb, but with added mag...
QUESTIONS?
Deja vu for websitesCACHING
Cache SystemsCACHE_BACKEND = ....   – Dummy caching      • ‘dummy:///’      • For development, dummy!   – Local-Memory Cac...
Approaches to Caching• Per site caching (generic approach)   – Only cache everything without GET or POST parameters   – On...
MIDDLEWARE_CLASSES = (       django.middleware.cache.UpdateCacheMiddleware,       django.middleware.common.CommonMiddlewar...
Approaches to Caching• Per site caching (generic approach)   – Only cache everything without GET or POST parameters   – On...
@cache_page(60 * 15)def my_view(request):    # ...
Approaches to Caching• Per site caching (generic approach)   – Only cache everything without GET or POST parameters   – On...
Approaches to Caching• Per site caching (generic approach)   – Only cache everything without GET or POST parameters   – On...
{% load cache %}{% cache 500 sidebar request.user.username %}  .. sidebar for logged in user ..{% endcache %}
Python & Django TTT
Python & Django TTT
Upcoming SlideShare
Loading in …5
×

Python & Django TTT

1,831 views
1,612 views

Published on

Python & Django TTT

  1. 1. @KevinVanWilder
  2. 2. Virtualenv• Isolated Python environment• Virtualenvwrapper $ which python /home/kevin/.virtualenvs/inuittest/bin/python $ pip install virtualenvwrapper $ mkproject inuittest ... (inuittest)$ which python /home/kevin/.virtualenvs/inuittest/bin/python (inuittest)$ deactivate $ workon inuittest
  3. 3. Installing libraries • Always work in a virtualenv$ pip freeze# ONE GAZILLION LIBRARIES!!!!(inuittest)$ pip freezeargparse==1.2.1wsgiref==0.1.2(inuittest)$ pip install yolk django south(inuittest)$ yolk –lDjango - 1.4.2 - activePython - 2.7.3 - active development (/usr/lib/python2.7/lib-dynload)South - 0.7.6 - activeargparse - 1.2.1 - active development (/usr/lib/python2.7)pip - 1.2.1 - activesetuptools - 0.6c11 - activewsgiref - 0.1.2 - active development (/usr/lib/python2.7)yolk - 0.4.3 - active
  4. 4. introduction@KevinVanWilder
  5. 5. Hi!• Architecture• Development Practices• The Django Ecosystem• Infrastructure• Tools
  6. 6. WTH is this thing?ARCHITECTURE OVERVIEW
  7. 7. Architecture RDBMS Storage ORM Fixtures ModelSignals Forms Template URL Resolver View Template Loader Middleware Middleware Request Response
  8. 8. Model RDBMS Storage ORM Fixtures ModelSignals Forms Template URL Resolver View Template Loader Middleware Middleware Request Response
  9. 9. Modelfrom django.db import modelsclass Poll(models.Model): question = models.CharField(max_length=200) pub_date = models.DateTimeField(date published)class Choice(models.Model): poll = models.ForeignKey(Poll) choice = models.CharField(max_length=200) votes = models.IntegerField()
  10. 10. ModelBEGIN;CREATE TABLE "polls_poll" ( "id" serial NOT NULL PRIMARY KEY, "question" varchar(200) NOT NULL, "pub_date" timestamp with time zone NOT NULL);CREATE TABLE "polls_choice" ( "id" serial NOT NULL PRIMARY KEY, "poll_id" integer NOT NULL REFERENCES "polls_poll" ("id") DEFERRABLE INITIALLYDEFERRED, "choice" varchar(200) NOT NULL, "votes" integer NOT NULL);COMMIT;
  11. 11. Request Response Cycle RDBMS Storage ORM Fixtures ModelSignals Forms Template URL Resolver View Template Loader Middleware Middleware Request Response
  12. 12. Request Response Cycle1. Determine which URLConf to use2. Load that URLConf and look for urlpatterns variable3. Run through each pattern in order and stop at the one that matches4. Once a match is found, import and call the view with the HttpRequest object5. If no regex matches or an exception is raised, invoke error-handling
  13. 13. Request Response Cycle: URLConf/polls /polls/1/polls/1/results /polls/1/votefrom django.conf.urls import patterns, include, urlfrom django.contrib import adminadmin.autodiscover()urlpatterns = patterns(, url(r^polls/$, polls.views.index), url(r^polls/(?P<poll_id>d+)/$, polls.views.detail), url(r^polls/(?P<poll_id>d+)/results/$, polls.views.results), url(r^polls/(?P<poll_id>d+)/vote/$, polls.views.vote), url(r^admin/, include(admin.site.urls)),) polls.views.detail(request=<HttpRequest object>, poll_id=‘1)
  14. 14. Request Response Cycle: Viewfrom django.shortcuts import render_to_responsefrom polls.models import Polldef index(request): latest_poll_list = Poll.objects.all().order_by(-pub_date)[:5] return render_to_response(polls/index.html, {latest_poll_list: latest_poll_list})
  15. 15. Request Response Cycle: Viewfrom django.http import Http404def detail(request, poll_id): p = get_object_or_404(Poll, pk=poll_id) return render_to_response(polls/detail.html, {poll: p})
  16. 16. Request Response Cycle: Viewfrom django.shortcuts import get_object_or_404, render_to_responsefrom django.http import HttpResponseRedirect, HttpResponsefrom django.core.urlresolvers import reversefrom django.template import RequestContextfrom polls.models import Choice, Polldef vote(request, poll_id): p = get_object_or_404(Poll, pk=poll_id) try: selected_choice = p.choice_set.get(pk=request.POST[choice]) except (KeyError, Choice.DoesNotExist): return render_to_response(polls/detail.html, { poll: p, error_message: "You didnt select a choice.", }, context_instance=RequestContext(request)) else: selected_choice.votes += 1 selected_choice.save() return HttpResponseRedirect(reverse(polls.views.results, args=(p.id,)))
  17. 17. Templating RDBMS Storage ORM Fixtures ModelSignals Forms Template URL Resolver View Template Loader Middleware Middleware Request Response
  18. 18. Template{% if latest_poll_list %} <ul> {% for poll in latest_poll_list %} <li><a href="/polls/{{ poll.id }}/">{{ poll.question }}</a></li> {% endfor %} </ul>{% else %} <p>No polls are available.</p>{% endif %}
  19. 19. The django developer in its natural habitatDJANGO ECOSYSTEM
  20. 20. Django Debug Toolbar• https://github.com/dcramer/django-debug- toolbar• Must have for all your projects
  21. 21. South• Data schema migration – Automatic migration creation – Database independent – Migration conflict detection – Under source control$ ./manage schemamigration –-initial--- update models$ ./manage schemamigration --auto
  22. 22. Celery• Asynchronous task queue• Message Passing• Green threads• RabbitMQ, Redis, MongoDB, CouchDB, …
  23. 23. • Search for Django• Solr, ElasticSearch, Whoosh, Xapian
  24. 24. Sentry• getsentry.com• Exception & error aggregator• Open Source and SaaS options
  25. 25. Tastypie• tastypieapi.org• Webservice API framework• REST• HATEOAS Compliant• Alternative to django-piston
  26. 26. Pinax• pinaxproject.com• Standard project layout• Starter projects• Default templates for prototyping• Reusable apps collection – Get ready...
  27. 27. Pinax Reusable Apps• Agon (points & positions) • Partitions (querysets)• Agon-ratings • Reminders• Agora (forum) • Rubberstamp (permissions)• Aiteo (q&a, cfr. Stackoverflow) • Timezones• Anafero (referrals) • Wakawaka (wiki)• Atom-format • Idios (profile app)• Biblion (blog) • Kaleo (user to user join invites)• Brabeion (badges) • Mailout (mail campaigns)• Dialogos (flaggable comments) • Marturion (testimonials)• announcements • Metron (analytics & metrics)• Boxes (db driven regions) • Minesis (media manager)• Contacts-import • Nashvegas (db migration)• Email-confirmation • Phileo (likes)• Flag (inappropriate/spam) • Pinax (rapid development)• Friends (contact/invitation) • Pinax-shop• Mailer (queuing) • Pinax-theme-bootstrap• Notification • Pinax-theme-classic• Oauth-access • Symposion (apps for conferences)• Observer
  28. 28. django-extensions• Shell_plus – Autoloading of the apps database models• Additional database fields – AutoSlugField – CreationDateTimeField – UUIDField – EncryptedCharField• DB Model graphing• …
  29. 29. INFRASTRUCTURE
  30. 30. Application Server (AS)Web Server Gateway Interface (WSGI)• PEP 333 [P. J. Eby, dec 2010]• Common ground for portable web applications – Routing, environment variables, load balancing, forwarding, post-prosessing• 2 parts: – Gateway (= server) – Framework (= application)
  31. 31. AS: gunicorn• “green unicorn” (cfr. green threading)• Pre-fork worker model• 4-12 workers = 1000’s req/sec• Each worker runs an instance of your django project => memory
  32. 32. AS: uWSGI• Protocol “uwsgi” (lowercase)• Included in Nginx & Cherokee• Faster than gunicorn
  33. 33. TOOLS
  34. 34. Tools• IPython – Tab completion in the interpreter!• import ipdb; ipdb.set_trace() – Just like pdb, but with added magic• Fabric (fabfile.org) – $ fab production deploy – Like Capistrano but without the voodoo• Supervisord (supervisord.org) – Process monitoring & controlling
  35. 35. QUESTIONS?
  36. 36. Deja vu for websitesCACHING
  37. 37. Cache SystemsCACHE_BACKEND = .... – Dummy caching • ‘dummy:///’ • For development, dummy! – Local-Memory Caching • ‘locmem:///’ • In memory caching without memcache – Database Caching • ‘db://my_cache_table’ • When DB I/O is not a problem. – Filesystem Caching • ‘file:///var/tmp/django_cache’ # directory • Cache values stored in seperate files in pickled format – Memcache • ‘memcached:///127.0.0.1:112211/’
  38. 38. Approaches to Caching• Per site caching (generic approach) – Only cache everything without GET or POST parameters – Only cache anonymous requests• Per view caching (granular approach) – Cache response of a view – Set cache timeouts per view• Template Fragment caching (micro manage approach) – Caching both static and dynamic fragments• Low level caching (micro micro manage) – Indivudually store parameters in the cache
  39. 39. MIDDLEWARE_CLASSES = ( django.middleware.cache.UpdateCacheMiddleware, django.middleware.common.CommonMiddleware, django.middleware.cache.FetchFromCacheMiddleware,)CACHE_MIDDLEWARE_ANONYMOUS_ONLY = True # optional
  40. 40. Approaches to Caching• Per site caching (generic approach) – Only cache everything without GET or POST parameters – Only cache anonymous requests• Per view caching (granular approach) – Cache response of a view – Set cache timeouts per view• Template Fragment caching (micro manage approach) – Caching both static and dynamic fragments• Low level caching (micro micro manage) – Indivudually store parameters in the cache
  41. 41. @cache_page(60 * 15)def my_view(request): # ...
  42. 42. Approaches to Caching• Per site caching (generic approach) – Only cache everything without GET or POST parameters – Only cache anonymous requests• Per view caching (granular approach) – Cache response of a view – Set cache timeouts per view• Template Fragment caching (micro manage approach) – Caching both static and dynamic fragments• Low level caching (micro micro manage) – Indivudually store parameters in the cache
  43. 43. Approaches to Caching• Per site caching (generic approach) – Only cache everything without GET or POST parameters – Only cache anonymous requests• Per view caching (granular approach) – Cache response of a view – Set cache timeouts per view• Template Fragment caching (micro manage approach) – Caching both static and dynamic fragments• Low level caching (micro micro manage) – Indivudually store parameters in the cache
  44. 44. {% load cache %}{% cache 500 sidebar request.user.username %} .. sidebar for logged in user ..{% endcache %}

×