Large problems, Mostly Solved

Large Problems Mostly Solved Eric Holscher http://ericholscher.com Djangocon 2010 Wednesday, September 8, 2010

What this talk isn’t » In depth » Overly technical » Conclusive Wednesday, September 8, 2010

What this talk is » Providing current best-of-class solutions to common problems » Identifying problems that aren’t currently solved well Wednesday, September 8, 2010

Overview » Patterns among good apps » The problems that are mostly solved » Unsolved Problems Wednesday, September 8, 2010

A good application Wednesday, September 8, 2010

Leverages previous knowledge/APIs Wednesday, September 8, 2010

Easy Setup Wednesday, September 8, 2010

Good upgrade path Wednesday, September 8, 2010

Good documentation Wednesday, September 8, 2010

Well Tested Wednesday, September 8, 2010

Solves a real problem Wednesday, September 8, 2010

Mostly Solved Problems Wednesday, September 8, 2010

Search Wednesday, September 8, 2010

Haystack » 9782 lines of docs » 8882 lines of tests » http://haystacksearch.org/ » Written by Daniel Lindsley Wednesday, September 8, 2010

Line counts ﬁnd . |egrep "rst|txt" |xargs wc -l |cut -c1-8 | awk '{s+=$1} END {print s}' ﬁnd . |egrep "py" |xargs wc -l |cut -c1-8 | awk '{s+=$1} END {print s}' #FORE! Wednesday, September 8, 2010

Setup » pip install django-haystack » Add ‘haystack’ to Installed Apps » Settings for siteconf and backend » Create Search Site » Create Search Indexes » Add views » Reindex Wednesday, September 8, 2010

Upgrade Path » Simple backend » Whoosh » Solr Wednesday, September 8, 2010

Beautiful APIs unfriendly_results = SearchQuerySet().exclude (content='hello').ﬁlter(content='world') unfriendly_results.order_by('-pub_date')[:5] Wednesday, September 8, 2010

Beautiful APIs class NoteIndex(SearchIndex): text = CharField(document=True, use_template=True) author = CharField(model_attr='user') pub_date = DateTimeField(model_attr='pub_date') def get_queryset(self): return Note.objects.ﬁlter (pub_date__lte=datetime.datetime.now()) Wednesday, September 8, 2010

Awesome Features » Faceting » Highlighting » More Like This » Easy Customization Wednesday, September 8, 2010

Wednesday, September 8, 2010

Amazing full-text search Wednesday, September 8, 2010

In an hour. Wednesday, September 8, 2010

Documentation Wednesday, September 8, 2010

Sphinx » 16598 lines of docs » 13272 lines of tests » http://sphinx.pocoo.org/ » Written by Georg Brandl Wednesday, September 8, 2010

Setup » pip install Sphinx » sphinx-quickstart » Edit/Create Restructured Text ﬁles » make html Wednesday, September 8, 2010

Awesome features » Uses ReStructuredText » Easily generate PDFs » Link between your own and other docs » Themes Wednesday, September 8, 2010

Shameless Self Promotion » Use Read The Docs » Host Sphinx docs, or create them there. » http://rtfd.org » http://media.readthedocs.org/ django.pdf Wednesday, September 8, 2010

Make Beautiful Documentation With Tools You Know Wednesday, September 8, 2010

Database Migrations Wednesday, September 8, 2010

South » 5662 lines of docs » 4328 lines of tests » http://south.aeracode.org/ » Written by Andrew Godwin Wednesday, September 8, 2010

Setup » pip install south » south in Installed Apps » manage.py schemamigration app -- initial » manage.py schemamigratoin app name --auto » manage.py migrate Wednesday, September 8, 2010

Beautiful API def forwards(self, orm): for adopter in orm.Adopter.objects.all(): try: adopter.ﬁrst_name, adopter.last_name = adopter.name.split() except ValueError: adopter.ﬁrst_name, adopter.last_name = adopter.name, "" adopter.save() Wednesday, September 8, 2010

Boring Wednesday, September 8, 2010

Safe, Painless Data Migration Wednesday, September 8, 2010

Delayed Execution Wednesday, September 8, 2010

Run commands out of process Wednesday, September 8, 2010

Celery » 19610 lines of docs » 13822 lines of tests » http://celeryproject.org/ » Written by Ask Solem Wednesday, September 8, 2010

Setup » pip install django-celery » Add ‘djcelery’ to Installed Apps » Syncdb » Conﬁgure your settings backend » Add @task decorator to function Wednesday, September 8, 2010

When and Why » Makes your site fast » Use for tasks that users don’t immediately care about » Even if they do, return fast to a waiting page Wednesday, September 8, 2010

Examples » Activity » Sending Email » Complex DB queries Wednesday, September 8, 2010

Path » Start with ALWAYS_EAGER » Use a ghetto queue » Rabbit MQ Wednesday, September 8, 2010

Awesome features » Run on multiple machines » Cron replacement » Good error reporting Wednesday, September 8, 2010

Magically make your code async Wednesday, September 8, 2010

Remote Execution Wednesday, September 8, 2010

Run commands on remote servers (for deployment) Wednesday, September 8, 2010

Fabric » 7272 lines of docs » 4572 lines of tests » http://fabﬁle.org » Written by Christian Vest Hansen & Jeff Forcier Wednesday, September 8, 2010

Setup » pip install fabric » Add commands to a fabﬁle.py » fab <command> Wednesday, September 8, 2010

Simple example #Fabﬁle from fabric.api import run def host_type(): run('uname -s') # Command $ fab -H example.com host_type Wednesday, September 8, 2010

Write python to deploy your code Wednesday, September 8, 2010

Deployment Wednesday, September 8, 2010

Gunicorn » 1898 lines of docs » 2172 lines of tests » http://gunicorn.org/ » Created by Benoit Chesneau Wednesday, September 8, 2010

Setup » pip install gunicorn » add gunicorn to your Installed Apps » manage.py run_gunicorn Wednesday, September 8, 2010

Need to install Nginx in front Wednesday, September 8, 2010

Nginx setup server { listen 80; server_name example.com; access_log /var/log/nginx/example.log; location / { proxy_pass http://127.0.0.1:8080; } } Wednesday, September 8, 2010

Awesome Features » Async backend workers » Simple conﬁguration » Python Wednesday, September 8, 2010

Stupidly Simple, Production Ready Deployment Wednesday, September 8, 2010

Packaging Wednesday, September 8, 2010

Pip & Virtualenv » Written by Ian Bicking » http://pip-installer.org » http://virtualenv.openplans.org Wednesday, September 8, 2010

Set up » sudo easy_install pip # :D » sudo pip install virtualenv » virtualenv awesome_env » source awesome_env/bin/activate » pip install Django Wednesday, September 8, 2010

Awesome Features » Separate environments for each application or deployment » Install code from repos » Doesn’t use eggs » User-owned environment Wednesday, September 8, 2010

Adds community standards to something everyone was doing their own way Wednesday, September 8, 2010

No more PYTHONPATH hacking Wednesday, September 8, 2010

Competing Applications Wednesday, September 8, 2010

APIs Wednesday, September 8, 2010

APIs » TastyPie » Piston Wednesday, September 8, 2010

Tagging Wednesday, September 8, 2010

Tagging » Django Taggit * » Django Tagging Wednesday, September 8, 2010

Continuous Integration Wednesday, September 8, 2010

Continuous Integration » Hudson * » Buildbot Wednesday, September 8, 2010

Smaller Problems Wednesday, September 8, 2010

Debugging in Development Wednesday, September 8, 2010

django-debug-toolbar » http://github.com/robhudson/ django-debug-toolbar/ » Written by Rob Hudson Wednesday, September 8, 2010

Proﬁles & Registration Wednesday, September 8, 2010

django-[proﬁles|registration] » http://bitbucket.org/ubernostrum/ django-proﬁles/ » http://bitbucket.org/ubernostrum/ django-registration/ » Written by James Bennett Wednesday, September 8, 2010

Filtering Wednesday, September 8, 2010

django-ﬁlter » http://github.com/alex/django-ﬁlter » Written by Alex Gaynor Wednesday, September 8, 2010

Finding packages Wednesday, September 8, 2010

Djangopackages.com Wednesday, September 8, 2010

Unsolved Problems Wednesday, September 8, 2010

Template Tags Wednesday, September 8, 2010

Logging Wednesday, September 8, 2010

Model Introspection Wednesday, September 8, 2010

Class Based Views Wednesday, September 8, 2010

Source code arrangement Wednesday, September 8, 2010

Notiﬁcations/Email Wednesday, September 8, 2010

Debugging in Production Wednesday, September 8, 2010

OpenID/OAuth Wednesday, September 8, 2010

VCS Abstraction Wednesday, September 8, 2010

Positive Note Wednesday, September 8, 2010

Cal’s list of problems ’08 » Multiple Databases » Multiple multi-keyed cache requests » Slow HTTP requests while we're doing other stuff » ORM creates dumb SQL » Verbose template syntax » No query debugger » Ugly SQL syntax » Commit all ﬁelds at once » No mascot » No deployment system » Migrating Models Wednesday, September 8, 2010

Cal’s list of problems ’08 » Multiple Databases » Multiple multi-keyed cache requests » Slow HTTP requests while we're doing other stuff » ORM creates dumb SQL » Verbose template syntax » No query debugger » Ugly SQL syntax (Table aliases) » Commit all ﬁelds at once » No mascot » No deployment system » Extending models Wednesday, September 8, 2010

Remember » Leverage good applications in the community » Take previous knowledge and apply it to new situations » You can create an awesome application used by other people Wednesday, September 8, 2010

Questions? » eric@ericholscher.com » twitter.com/ericholscher Wednesday, September 8, 2010

http://ericholscher.com	695
http://lanyrd.com	6
https://team.nwebs.de	4
http://localhost	2
http://dashboard.bloglines.com	2
http://localhost.dev:8000	1
http://www.ericholscher.com	1
http://www.rtfd.org	1
http://static.slidesharecdn.com	1
http://translate.googleusercontent.com	1

Large problems, Mostly Solved

by ericholscher on Sep 08, 2010

Accessibility

Tags

Upload Details

Usage Rights

10 Embeds 714

Statistics

Large problems, Mostly Solved — Presentation Transcript