Django: Two Extreme
   Case Studies
      Mike Biglan: CTO
  Wiggins: Senior Developer


         Concentric Sky
         ...
Overview

1. About the Projects
2. Speed of Development
3. Standards & Bootstrapping
4. Scaling
5. Integrations
About Concentric Sky

• Web, Mobile, Enterprise Java Development
• 35 people in Eugene, OR
• Our Django team has almost hi...
Two Extreme Deadlines



• MichaelMoore.com: 5 weeks
• Santa Fe Institute (santafe.edu):6
  weeks
Overview

1. About the Projects
2. Speed of Development
3. Standards & Bootstrapping
4. Scaling
5. Integrations
Process


• Owners of well-defined tasks
• Blockers identified/resolved ASAP
• Daily/routine checkins
• As issues come up, i...
Planning
• Data Schema (ERD) -> Django models
• Don’t overplan
• Vision to client/developers: sitemap, mockups,
  etc
 • O...
Content & Migration

• Content components identified
 • For each, owner for creating/moving
   content
• Structured content...
Standards


• Standard project structure
• Standard project lifecycle
 • Predict the questions/blockers as
   early as pos...
Overview

1. About the Projects
2. Speed of Development
3. Standards & Bootstrapping
4. Scaling
5. Integrations
Bootstrapping:
        Traditional


• django-admin.py startproject
• clone latest project & prune
• clone a pre-pruned te...
Standard Project
    Structure: Why?

• Developers new to Django
• Developers new to project
• Easier to deploy, easier to...
Djenesis
 • Bootstraps a new project
   using a template
 • Included: well-formed,
   web-based project
   template



htt...
Directory Structure

• apps: this project’s apps (in PYTHONPATH)
• lib: 3rd party/helper libraries (in
  PYTHONPATH)

• et...
Mainsite

• manage.py: setup additional paths (top/
  lib/app)
• settings.py: primary settings, loads local
• local_settin...
Local Settings

• Multiple machines
 • Multiple people
 • Local, review, live servers
• Machine-dependent go in local_sett...
Overview

1. About the Projects
2. Speed of Development
3. Standards & Bootstrapping
4. Scaling
5. Integrations
Deploying Django




                  Text




http://www.djangobook.com/en/2.0/chapter12/
X-treme proxying
• Separate media server(s)
• Proxy dedicated database server(s)
• Load Balance app server(s)
• Use memcac...
CacheModel
• Quick, easy, does the heavy lifting

• Namespaced caching library

• Table-level caching with cache_key()

• ...
Book & Authors


     Book



    Author
Lets make some cache
from django.db import models
from django.core.cache import cache
class BookManager(models.Manager):
 ...
Lets make some cache
from django.db import models
from django.core.cache import cache
class BookManager(models.Manager):
 ...
Lets make some cache
from django.db import models
from django.core.cache import cache
class BookManager(models.Manager):
 ...
Gimme da cache
from django.db import models
from cachemodel import models as cache_models
class Book(cache_models.CacheMod...
Object Level Caching
from django.db import models
from cachemodel import models as cache_models
class Book(cache_models.Ca...
Overview

1. About the Projects
2. Speed of Development
3. Standards & Bootstrapping
4. Scaling
5. System Integrations
System Integrations


• Google Search Appliance (GSA)
• Alfresco
• iCal Server
• Haystack/SOLR
Integrations


• 1) Identify (a) service and (b) push or
  pull
• 2a) Pull: Make a proxy model
 • Use Caching
• 2b) Push: ...
Upcoming SlideShare
Loading in …5
×

Djenesis & CacheModel for Django - Oscon2010

3,341 views

Published on

We were fortunate this past year to develop two of the larger Django applications out there – in the span of 12 weeks: michaelmoore.com and Santa Fe Institute's santafe.edu. Between the two, these sites have multiple layers of memcached caching, multiple web servers and database servers, integrated site search (Lucene/SOLR and Google GSA), DjangoCMS, and integrations with iCal and Alfresco.

Published in: Technology
0 Comments
12 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,341
On SlideShare
0
From Embeds
0
Number of Embeds
57
Actions
Shares
0
Downloads
0
Comments
0
Likes
12
Embeds 0
No embeds

No notes for slide





























  • Djenesis & CacheModel for Django - Oscon2010

    1. Django: Two Extreme Case Studies Mike Biglan: CTO Wiggins: Senior Developer Concentric Sky www.concentricsky.com
    2. Overview 1. About the Projects 2. Speed of Development 3. Standards & Bootstrapping 4. Scaling 5. Integrations
    3. About Concentric Sky • Web, Mobile, Enterprise Java Development • 35 people in Eugene, OR • Our Django team has almost hit 10 people • Django projects go well • People enjoy using it
    4. Two Extreme Deadlines • MichaelMoore.com: 5 weeks • Santa Fe Institute (santafe.edu):6 weeks
    5. Overview 1. About the Projects 2. Speed of Development 3. Standards & Bootstrapping 4. Scaling 5. Integrations
    6. Process • Owners of well-defined tasks • Blockers identified/resolved ASAP • Daily/routine checkins • As issues come up, incorporate them into global process
    7. Planning • Data Schema (ERD) -> Django models • Don’t overplan • Vision to client/developers: sitemap, mockups, etc • On same page? • Early identification of changes; least costly • Website Specification Document: Spreadsheet
    8. Content & Migration • Content components identified • For each, owner for creating/moving content • Structured content much easier • Migration Methods: Python script, SQL script, manual to review site
    9. Standards • Standard project structure • Standard project lifecycle • Predict the questions/blockers as early as possible
    10. Overview 1. About the Projects 2. Speed of Development 3. Standards & Bootstrapping 4. Scaling 5. Integrations
    11. Bootstrapping: Traditional • django-admin.py startproject • clone latest project & prune • clone a pre-pruned template, update several variables
    12. Standard Project Structure: Why? • Developers new to Django • Developers new to project • Easier to deploy, easier to build deployment scripts & environments • Libs can be externals/sub-modules; isolated within each project
    13. Djenesis • Bootstraps a new project using a template • Included: well-formed, web-based project template http://code.google.com/p/djenesis/
    14. Directory Structure • apps: this project’s apps (in PYTHONPATH) • lib: 3rd party/helper libraries (in PYTHONPATH) • etc: wsgi template file • mainsite: the “main” project site • media: css, img, js • templates: with simple base.html
    15. Mainsite • manage.py: setup additional paths (top/ lib/app) • settings.py: primary settings, loads local • local_settings.py: machine-dependent settings • urls.py: In debug, serves static media; admin uncommented by default
    16. Local Settings • Multiple machines • Multiple people • Local, review, live servers • Machine-dependent go in local_settings • Not checked in to version control
    17. Overview 1. About the Projects 2. Speed of Development 3. Standards & Bootstrapping 4. Scaling 5. Integrations
    18. Deploying Django Text http://www.djangobook.com/en/2.0/chapter12/
    19. X-treme proxying • Separate media server(s) • Proxy dedicated database server(s) • Load Balance app server(s) • Use memcached for sticky sessions • Cache your queries
    20. CacheModel • Quick, easy, does the heavy lifting • Namespaced caching library • Table-level caching with cache_key() • Object-level caching with ns_cache_key() http://code.google.com/p/django-cachemodel/
    21. Book & Authors Book Author
    22. Lets make some cache from django.db import models from django.core.cache import cache class BookManager(models.Manager):     def get_by_slug(self, slug, cache_timeout=900):         cache_key = "book_by_slug_%s" % (slug,)         book = cache.get(cache_key)         if book is None:             book = Book.objects.get(slug=slug)             cache.set(cache_key, book, cache_timeout)         return book class Book(models.Model):     slug = models.SlugField(max_length=128)     name = models.CharField(max_length=128)     objects = BookManager()     def save(self, *args, **kwargs):         super(Book, self).save(*args, **kwargs)         cache.delete("book_by_slug_%s" % (self.slug,))     def delete(self, *args, **kwargs):         super(Book, self).save(*args, **kwargs)         cache.delete("book_by_slug_%s" % (self.slug,)) >>> Book.objects.get_by_slug("my-book-slug") <Book: Book Object>
    23. Lets make some cache from django.db import models from django.core.cache import cache class BookManager(models.Manager):     def get_by_slug(self, slug, cache_timeout=900):         cache_key = "book_by_slug_%s" % (slug,)         book = cache.get(cache_key)         if book is None:             book = Book.objects.get(slug=slug)             cache.set(cache_key, book, cache_timeout)         return book class Book(models.Model):     slug = models.SlugField(max_length=128)     name = models.CharField(max_length=128)     objects = BookManager()     def save(self, *args, **kwargs):         super(Book, self).save(*args, **kwargs)         cache.delete("book_by_slug_%s" % (self.slug,))     def delete(self, *args, **kwargs):         super(Book, self).save(*args, **kwargs)         cache.delete("book_by_slug_%s" % (self.slug,)) >>> Book.objects.get_by_slug("my-book-slug") <Book: Book Object>
    24. Lets make some cache from django.db import models from django.core.cache import cache class BookManager(models.Manager):     def get_by_slug(self, slug, cache_timeout=900):         cache_key = "book_by_slug_%s" % (slug,)         book = cache.get(cache_key)         if book is None:             book = Book.objects.get(slug=slug)             cache.set(cache_key, book, cache_timeout)         return book class Book(models.Model):     slug = models.SlugField(max_length=128)     name = models.CharField(max_length=128)     objects = BookManager()     def save(self, *args, **kwargs):         super(Book, self).save(*args, **kwargs)         cache.delete("book_by_slug_%s" % (self.slug,))     def delete(self, *args, **kwargs):         super(Book, self).save(*args, **kwargs)         cache.delete("book_by_slug_%s" % (self.slug,)) >>> Book.objects.get_by_slug("my-book-slug") <Book: Book Object>
    25. Gimme da cache from django.db import models from cachemodel import models as cache_models class Book(cache_models.CacheModel):     slug = models.SlugField(max_length=128)     name = models.CharField(max_length=128) >>> Book.objects.get_by("slug", "my-book-slug") <Book: Book Object>
    26. Object Level Caching from django.db import models from cachemodel import models as cache_models class Book(cache_models.CacheModel):     slug = models.SlugField(max_length=128)     name = models.CharField(max_length=128)     @cache_models.cached_method(900, 'authors')     def get_authors(self):         return self.bookauthor_set.all() class BookAuthor(cache_models.CacheModel):     book = models.ForeignKey(Book)     name = models.CharField(max_length=255)     def flush_cache(self):         super(BookAuthor, self).flush_cache()         self.book.flush_cache()
    27. Overview 1. About the Projects 2. Speed of Development 3. Standards & Bootstrapping 4. Scaling 5. System Integrations
    28. System Integrations • Google Search Appliance (GSA) • Alfresco • iCal Server • Haystack/SOLR
    29. Integrations • 1) Identify (a) service and (b) push or pull • 2a) Pull: Make a proxy model • Use Caching • 2b) Push: Make a web service

    ×