Large problems, Mostly Solved

  • 10,461 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
10,461
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
203
Comments
0
Likes
25

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Large Problems Mostly Solved Eric Holscher http://ericholscher.com Djangocon 2010 Wednesday, September 8, 2010
  • 2. What this talk isn’t » In depth » Overly technical » Conclusive Wednesday, September 8, 2010
  • 3. What this talk is » Providing current best-of-class solutions to common problems » Identifying problems that aren’t currently solved well Wednesday, September 8, 2010
  • 4. Overview » Patterns among good apps » The problems that are mostly solved » Unsolved Problems Wednesday, September 8, 2010
  • 5. A good application Wednesday, September 8, 2010
  • 6. Leverages previous knowledge/APIs Wednesday, September 8, 2010
  • 7. Easy Setup Wednesday, September 8, 2010
  • 8. Good upgrade path Wednesday, September 8, 2010
  • 9. Good documentation Wednesday, September 8, 2010
  • 10. Well Tested Wednesday, September 8, 2010
  • 11. Solves a real problem Wednesday, September 8, 2010
  • 12. Mostly Solved Problems Wednesday, September 8, 2010
  • 13. Search Wednesday, September 8, 2010
  • 14. Haystack » 9782 lines of docs » 8882 lines of tests » http://haystacksearch.org/ » Written by Daniel Lindsley Wednesday, September 8, 2010
  • 15. Line counts find . |egrep "rst|txt" |xargs wc -l |cut -c1-8 | awk '{s+=$1} END {print s}' find . |egrep "py" |xargs wc -l |cut -c1-8 | awk '{s+=$1} END {print s}' #FORE! Wednesday, September 8, 2010
  • 16. Setup » pip install django-haystack » Add ‘haystack’ to Installed Apps » Settings for siteconf and backend » Create Search Site » Create Search Indexes » Add views » Reindex Wednesday, September 8, 2010
  • 17. Upgrade Path » Simple backend » Whoosh » Solr Wednesday, September 8, 2010
  • 18. Beautiful APIs unfriendly_results = SearchQuerySet().exclude (content='hello').filter(content='world') unfriendly_results.order_by('-pub_date')[:5] Wednesday, September 8, 2010
  • 19. Beautiful APIs class NoteIndex(SearchIndex): text = CharField(document=True, use_template=True) author = CharField(model_attr='user') pub_date = DateTimeField(model_attr='pub_date') def get_queryset(self): return Note.objects.filter (pub_date__lte=datetime.datetime.now()) Wednesday, September 8, 2010
  • 20. Awesome Features » Faceting » Highlighting » More Like This » Easy Customization Wednesday, September 8, 2010
  • 21. Wednesday, September 8, 2010
  • 22. Amazing full-text search Wednesday, September 8, 2010
  • 23. In an hour. Wednesday, September 8, 2010
  • 24. Documentation Wednesday, September 8, 2010
  • 25. Sphinx » 16598 lines of docs » 13272 lines of tests » http://sphinx.pocoo.org/ » Written by Georg Brandl Wednesday, September 8, 2010
  • 26. Setup » pip install Sphinx » sphinx-quickstart » Edit/Create Restructured Text files » make html Wednesday, September 8, 2010
  • 27. Awesome features » Uses ReStructuredText » Easily generate PDFs » Link between your own and other docs » Themes Wednesday, September 8, 2010
  • 28. Shameless Self Promotion » Use Read The Docs » Host Sphinx docs, or create them there. » http://rtfd.org » http://media.readthedocs.org/ django.pdf Wednesday, September 8, 2010
  • 29. Wednesday, September 8, 2010
  • 30. Make Beautiful Documentation With Tools You Know Wednesday, September 8, 2010
  • 31. Database Migrations Wednesday, September 8, 2010
  • 32. South » 5662 lines of docs » 4328 lines of tests » http://south.aeracode.org/ » Written by Andrew Godwin Wednesday, September 8, 2010
  • 33. Setup » pip install south » south in Installed Apps » manage.py schemamigration app -- initial » manage.py schemamigratoin app name --auto » manage.py migrate Wednesday, September 8, 2010
  • 34. Beautiful API def forwards(self, orm): for adopter in orm.Adopter.objects.all(): try: adopter.first_name, adopter.last_name = adopter.name.split() except ValueError: adopter.first_name, adopter.last_name = adopter.name, "" adopter.save() Wednesday, September 8, 2010
  • 35. Boring Wednesday, September 8, 2010
  • 36. Safe, Painless Data Migration Wednesday, September 8, 2010
  • 37. Delayed Execution Wednesday, September 8, 2010
  • 38. Run commands out of process Wednesday, September 8, 2010
  • 39. Celery » 19610 lines of docs » 13822 lines of tests » http://celeryproject.org/ » Written by Ask Solem Wednesday, September 8, 2010
  • 40. Setup » pip install django-celery » Add ‘djcelery’ to Installed Apps » Syncdb » Configure your settings backend » Add @task decorator to function Wednesday, September 8, 2010
  • 41. When and Why » Makes your site fast » Use for tasks that users don’t immediately care about » Even if they do, return fast to a waiting page Wednesday, September 8, 2010
  • 42. Examples » Activity » Sending Email » Complex DB queries Wednesday, September 8, 2010
  • 43. Wednesday, September 8, 2010
  • 44. Path » Start with ALWAYS_EAGER » Use a ghetto queue » Rabbit MQ Wednesday, September 8, 2010
  • 45. Awesome features » Run on multiple machines » Cron replacement » Good error reporting Wednesday, September 8, 2010
  • 46. Magically make your code async Wednesday, September 8, 2010
  • 47. Remote Execution Wednesday, September 8, 2010
  • 48. Run commands on remote servers (for deployment) Wednesday, September 8, 2010
  • 49. Fabric » 7272 lines of docs » 4572 lines of tests » http://fabfile.org » Written by Christian Vest Hansen & Jeff Forcier Wednesday, September 8, 2010
  • 50. Setup » pip install fabric » Add commands to a fabfile.py » fab <command> Wednesday, September 8, 2010
  • 51. Simple example #Fabfile from fabric.api import run def host_type(): run('uname -s') # Command $ fab -H example.com host_type Wednesday, September 8, 2010
  • 52. Write python to deploy your code Wednesday, September 8, 2010
  • 53. Deployment Wednesday, September 8, 2010
  • 54. Gunicorn » 1898 lines of docs » 2172 lines of tests » http://gunicorn.org/ » Created by Benoit Chesneau Wednesday, September 8, 2010
  • 55. Setup » pip install gunicorn » add gunicorn to your Installed Apps » manage.py run_gunicorn Wednesday, September 8, 2010
  • 56. Need to install Nginx in front Wednesday, September 8, 2010
  • 57. Nginx setup server { listen 80; server_name example.com; access_log /var/log/nginx/example.log; location / { proxy_pass http://127.0.0.1:8080; } } Wednesday, September 8, 2010
  • 58. Awesome Features » Async backend workers » Simple configuration » Python Wednesday, September 8, 2010
  • 59. Stupidly Simple, Production Ready Deployment Wednesday, September 8, 2010
  • 60. Packaging Wednesday, September 8, 2010
  • 61. Pip & Virtualenv » Written by Ian Bicking » http://pip-installer.org » http://virtualenv.openplans.org Wednesday, September 8, 2010
  • 62. Set up » sudo easy_install pip # :D » sudo pip install virtualenv » virtualenv awesome_env » source awesome_env/bin/activate » pip install Django Wednesday, September 8, 2010
  • 63. Awesome Features » Separate environments for each application or deployment » Install code from repos » Doesn’t use eggs » User-owned environment Wednesday, September 8, 2010
  • 64. Adds community standards to something everyone was doing their own way Wednesday, September 8, 2010
  • 65. No more PYTHONPATH hacking Wednesday, September 8, 2010
  • 66. Competing Applications Wednesday, September 8, 2010
  • 67. APIs Wednesday, September 8, 2010
  • 68. APIs » TastyPie » Piston Wednesday, September 8, 2010
  • 69. Tagging Wednesday, September 8, 2010
  • 70. Tagging » Django Taggit * » Django Tagging Wednesday, September 8, 2010
  • 71. Continuous Integration Wednesday, September 8, 2010
  • 72. Continuous Integration » Hudson * » Buildbot Wednesday, September 8, 2010
  • 73. Smaller Problems Wednesday, September 8, 2010
  • 74. Debugging in Development Wednesday, September 8, 2010
  • 75. django-debug-toolbar » http://github.com/robhudson/ django-debug-toolbar/ » Written by Rob Hudson Wednesday, September 8, 2010
  • 76. Wednesday, September 8, 2010
  • 77. Profiles & Registration Wednesday, September 8, 2010
  • 78. django-[profiles|registration] » http://bitbucket.org/ubernostrum/ django-profiles/ » http://bitbucket.org/ubernostrum/ django-registration/ » Written by James Bennett Wednesday, September 8, 2010
  • 79. Filtering Wednesday, September 8, 2010
  • 80. django-filter » http://github.com/alex/django-filter » Written by Alex Gaynor Wednesday, September 8, 2010
  • 81. Wednesday, September 8, 2010
  • 82. Finding packages Wednesday, September 8, 2010
  • 83. Djangopackages.com Wednesday, September 8, 2010
  • 84. Unsolved Problems Wednesday, September 8, 2010
  • 85. Template Tags Wednesday, September 8, 2010
  • 86. Logging Wednesday, September 8, 2010
  • 87. Model Introspection Wednesday, September 8, 2010
  • 88. Class Based Views Wednesday, September 8, 2010
  • 89. Source code arrangement Wednesday, September 8, 2010
  • 90. Notifications/Email Wednesday, September 8, 2010
  • 91. Debugging in Production Wednesday, September 8, 2010
  • 92. OpenID/OAuth Wednesday, September 8, 2010
  • 93. VCS Abstraction Wednesday, September 8, 2010
  • 94. Positive Note Wednesday, September 8, 2010
  • 95. Cal’s list of problems ’08 » Multiple Databases » Multiple multi-keyed cache requests » Slow HTTP requests while we're doing other stuff » ORM creates dumb SQL » Verbose template syntax » No query debugger » Ugly SQL syntax » Commit all fields at once » No mascot » No deployment system » Migrating Models Wednesday, September 8, 2010
  • 96. Cal’s list of problems ’08 » Multiple Databases » Multiple multi-keyed cache requests » Slow HTTP requests while we're doing other stuff » ORM creates dumb SQL » Verbose template syntax » No query debugger » Ugly SQL syntax (Table aliases) » Commit all fields at once » No mascot » No deployment system » Extending models Wednesday, September 8, 2010
  • 97. Remember » Leverage good applications in the community » Take previous knowledge and apply it to new situations » You can create an awesome application used by other people Wednesday, September 8, 2010
  • 98. Questions? » eric@ericholscher.com » twitter.com/ericholscher Wednesday, September 8, 2010