Djenesis & CacheModel for Django - Oscon2010

  • 2,008 views
Uploaded on

We were fortunate this past year to develop two of the larger Django applications out there – in the span of 12 weeks: michaelmoore.com and Santa Fe Institute's santafe.edu. Between the two, these …

We were fortunate this past year to develop two of the larger Django applications out there – in the span of 12 weeks: michaelmoore.com and Santa Fe Institute's santafe.edu. Between the two, these sites have multiple layers of memcached caching, multiple web servers and database servers, integrated site search (Lucene/SOLR and Google GSA), DjangoCMS, and integrations with iCal and Alfresco.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,008
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
0
Comments
0
Likes
12

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide





























Transcript

  • 1. Django: Two Extreme Case Studies Mike Biglan: CTO Wiggins: Senior Developer Concentric Sky www.concentricsky.com
  • 2. Overview 1. About the Projects 2. Speed of Development 3. Standards & Bootstrapping 4. Scaling 5. Integrations
  • 3. About Concentric Sky • Web, Mobile, Enterprise Java Development • 35 people in Eugene, OR • Our Django team has almost hit 10 people • Django projects go well • People enjoy using it
  • 4. Two Extreme Deadlines • MichaelMoore.com: 5 weeks • Santa Fe Institute (santafe.edu):6 weeks
  • 5. Overview 1. About the Projects 2. Speed of Development 3. Standards & Bootstrapping 4. Scaling 5. Integrations
  • 6. Process • Owners of well-defined tasks • Blockers identified/resolved ASAP • Daily/routine checkins • As issues come up, incorporate them into global process
  • 7. Planning • Data Schema (ERD) -> Django models • Don’t overplan • Vision to client/developers: sitemap, mockups, etc • On same page? • Early identification of changes; least costly • Website Specification Document: Spreadsheet
  • 8. Content & Migration • Content components identified • For each, owner for creating/moving content • Structured content much easier • Migration Methods: Python script, SQL script, manual to review site
  • 9. Standards • Standard project structure • Standard project lifecycle • Predict the questions/blockers as early as possible
  • 10. Overview 1. About the Projects 2. Speed of Development 3. Standards & Bootstrapping 4. Scaling 5. Integrations
  • 11. Bootstrapping: Traditional • django-admin.py startproject • clone latest project & prune • clone a pre-pruned template, update several variables
  • 12. Standard Project Structure: Why? • Developers new to Django • Developers new to project • Easier to deploy, easier to build deployment scripts & environments • Libs can be externals/sub-modules; isolated within each project
  • 13. Djenesis • Bootstraps a new project using a template • Included: well-formed, web-based project template http://code.google.com/p/djenesis/
  • 14. Directory Structure • apps: this project’s apps (in PYTHONPATH) • lib: 3rd party/helper libraries (in PYTHONPATH) • etc: wsgi template file • mainsite: the “main” project site • media: css, img, js • templates: with simple base.html
  • 15. Mainsite • manage.py: setup additional paths (top/ lib/app) • settings.py: primary settings, loads local • local_settings.py: machine-dependent settings • urls.py: In debug, serves static media; admin uncommented by default
  • 16. Local Settings • Multiple machines • Multiple people • Local, review, live servers • Machine-dependent go in local_settings • Not checked in to version control
  • 17. Overview 1. About the Projects 2. Speed of Development 3. Standards & Bootstrapping 4. Scaling 5. Integrations
  • 18. Deploying Django Text http://www.djangobook.com/en/2.0/chapter12/
  • 19. X-treme proxying • Separate media server(s) • Proxy dedicated database server(s) • Load Balance app server(s) • Use memcached for sticky sessions • Cache your queries
  • 20. CacheModel • Quick, easy, does the heavy lifting • Namespaced caching library • Table-level caching with cache_key() • Object-level caching with ns_cache_key() http://code.google.com/p/django-cachemodel/
  • 21. Book & Authors Book Author
  • 22. Lets make some cache from django.db import models from django.core.cache import cache class BookManager(models.Manager):     def get_by_slug(self, slug, cache_timeout=900):         cache_key = "book_by_slug_%s" % (slug,)         book = cache.get(cache_key)         if book is None:             book = Book.objects.get(slug=slug)             cache.set(cache_key, book, cache_timeout)         return book class Book(models.Model):     slug = models.SlugField(max_length=128)     name = models.CharField(max_length=128)     objects = BookManager()     def save(self, *args, **kwargs):         super(Book, self).save(*args, **kwargs)         cache.delete("book_by_slug_%s" % (self.slug,))     def delete(self, *args, **kwargs):         super(Book, self).save(*args, **kwargs)         cache.delete("book_by_slug_%s" % (self.slug,)) >>> Book.objects.get_by_slug("my-book-slug") <Book: Book Object>
  • 23. Lets make some cache from django.db import models from django.core.cache import cache class BookManager(models.Manager):     def get_by_slug(self, slug, cache_timeout=900):         cache_key = "book_by_slug_%s" % (slug,)         book = cache.get(cache_key)         if book is None:             book = Book.objects.get(slug=slug)             cache.set(cache_key, book, cache_timeout)         return book class Book(models.Model):     slug = models.SlugField(max_length=128)     name = models.CharField(max_length=128)     objects = BookManager()     def save(self, *args, **kwargs):         super(Book, self).save(*args, **kwargs)         cache.delete("book_by_slug_%s" % (self.slug,))     def delete(self, *args, **kwargs):         super(Book, self).save(*args, **kwargs)         cache.delete("book_by_slug_%s" % (self.slug,)) >>> Book.objects.get_by_slug("my-book-slug") <Book: Book Object>
  • 24. Lets make some cache from django.db import models from django.core.cache import cache class BookManager(models.Manager):     def get_by_slug(self, slug, cache_timeout=900):         cache_key = "book_by_slug_%s" % (slug,)         book = cache.get(cache_key)         if book is None:             book = Book.objects.get(slug=slug)             cache.set(cache_key, book, cache_timeout)         return book class Book(models.Model):     slug = models.SlugField(max_length=128)     name = models.CharField(max_length=128)     objects = BookManager()     def save(self, *args, **kwargs):         super(Book, self).save(*args, **kwargs)         cache.delete("book_by_slug_%s" % (self.slug,))     def delete(self, *args, **kwargs):         super(Book, self).save(*args, **kwargs)         cache.delete("book_by_slug_%s" % (self.slug,)) >>> Book.objects.get_by_slug("my-book-slug") <Book: Book Object>
  • 25. Gimme da cache from django.db import models from cachemodel import models as cache_models class Book(cache_models.CacheModel):     slug = models.SlugField(max_length=128)     name = models.CharField(max_length=128) >>> Book.objects.get_by("slug", "my-book-slug") <Book: Book Object>
  • 26. Object Level Caching from django.db import models from cachemodel import models as cache_models class Book(cache_models.CacheModel):     slug = models.SlugField(max_length=128)     name = models.CharField(max_length=128)     @cache_models.cached_method(900, 'authors')     def get_authors(self):         return self.bookauthor_set.all() class BookAuthor(cache_models.CacheModel):     book = models.ForeignKey(Book)     name = models.CharField(max_length=255)     def flush_cache(self):         super(BookAuthor, self).flush_cache()         self.book.flush_cache()
  • 27. Overview 1. About the Projects 2. Speed of Development 3. Standards & Bootstrapping 4. Scaling 5. System Integrations
  • 28. System Integrations • Google Search Appliance (GSA) • Alfresco • iCal Server • Haystack/SOLR
  • 29. Integrations • 1) Identify (a) service and (b) push or pull • 2a) Pull: Make a proxy model • Use Caching • 2b) Push: Make a web service