Django Pro ORM
Upcoming SlideShare
Loading in...5
×
 

Django Pro ORM

on

  • 6,222 views

 

Statistics

Views

Total Views
6,222
Views on SlideShare
6,181
Embed Views
41

Actions

Likes
20
Downloads
177
Comments
0

8 Embeds 41

http://www.linkedin.com 14
http://www.slideshare.net 9
http://lanyrd.com 7
https://www.linkedin.com 5
http://127.0.0.1 2
http://einztein.com 2
http://lanyrd.dev:8000 1
http://beta.einztein.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />

Django Pro ORM Django Pro ORM Presentation Transcript

  • Pro ORM Alex Gaynor @alex_gaynor Django NYC
  • What is this talk? ORM Architecture Practical Things 50% 50%
  • ORM Architecture
  • QuerySet-Refactor
  • • A big refactor of the ORM internals • Happened a bit before Django 1.0 Old New
  • Multi DB
  • • GSOC project this past summer • Added multiple database support to Django • Ripped up a lot of internals, and then put them back together Old New
  • from django.db import models class MyModel(models.Model): pass MyModel.objects Managers
  • class Manager(object): def get_query_set(self): return QuerySet(self.model) def get(self, *args, **kwargs): return self.get_query_set().get(*args, **kwargs) def filter(self, *args, **kwargs): return self.get_query_set().filter(*args, **kwargs) # ETC... get_query_set(), your gateway to the rabbit
  • QuerySet • Backend agnostic • Basically it handles turning the public API into calls to methods on Query • Surprisingly little meat for a 1400 line file • Also, a few subclasses for values(), values_list(), and dates()
  • Query
  • • This part is backend specific, SQL vs. GAE here, not Postgres vs. Oracle • It carries all the info around, ordering, joins, where, aggregates, etc... • It used to generate SQL, not anymore (more on this later)
  • self.model = model self.alias_refcount = {} self.alias_map = {} self.table_map = {} self.join_map = {} self.rev_join_map = {} self.quote_cache = {} self.default_cols = True self.default_ordering = True self.standard_ordering = True self.ordering_aliases = [] self.select_fields = [] self.related_select_fields = [] self.dupe_avoidance = {} self.used_aliases = set() self.filter_is_sticky = False self.included_inherited_models = {} self.select = [] self.tables = [] self.where = where() self.where_class = where self.group_by = None self.having = where()
  • self.order_by = [] self.low_mark, self.high_mark = 0, None self.distinct = False self.select_related = False self.related_select_cols = [] self.aggregates = SortedDict() self.aggregate_select_mask = None self._aggregate_select_cache = None self.max_depth = 5 self.extra = SortedDict() self.extra_select_mask = None self._extra_select_cache = None self.extra_tables = () self.extra_order_by = () self.deferred_loading = (set(), True)
  • I told you it was a lot of stuff
  • The Major Players Attributes Methods self.where self.get_compiler() self.having self.clone() self.aggregates self.join() self.alias_map self.add_filter() self.select_fields self.add_ordering()
  • Dude, where’s my SQL? There are no relevant images for SQL, so you get to settle for me trying to be funny
  • SQLCompiler • Takes Querys and turns ‘em into SQL. • Also executes SQL. • This handles the backend specific stuff. • We ship 2 sets of them. 1 for Oracle, and one for pretty much everything else.
  • 31 Flavors! SQLCompiler SQLInsertCompiler SQLDeleteCompiler SQLDateCompiler SQLUpdateCompiler SQLAggregateCompiler
  • django.db.backends.* • Interoperate with • Running the cmd line database drivers shell • Handle most of the • django.contrib.gis.db. nuances of the backends various databases’ SQL dialects • DDL for creation • Introspection
  • Putting this Stuff to Good Use aka the part you might use at your day job
  • Custom Aggregates
  • • Django comes with a few aggregates django.db.models.aggregates Avg • But databases have other Count aggregates Max • Some of them Min even let you StdDev create your own! Sum Variance
  • Aggregates Come in Two Parts • One part is a data carrier, it knows what field you’re aggregating over. • The other part turns that into SQL. • Starting to sound familiar?
  • Part I from django.db.models import Aggregate class MyAggregate(Aggregate): def add_to_query(self, query, alias, col, course, is_summary): aggregate = SQLMyAggregate(col, source=source, is_summary=is_summary, **extra) query.aggregates[alias] = aggregate
  • Part II from django.db.models.sql.aggregates import Aggregate as SQLAggregate class SQLMyAggregate(SQLAggregate): sql_template = "%(function)(cast(%(field)s as numeric) / 100)" sql_function = "MYFUNCTION" is_ordinal = True
  • Automatic Caching
  • Single Object Caching • Basically we want to automatically cache any get() lookups on unique fields (primary key, slugs, etc.) • These types of queries seem to pop up a lot (think every single foreign key traversal) • Also automatically do invalidation for us.
  • from django.core.cache import cache from django.db.models.query import QuerySet class CachingQuerySet(QuerySet): def get(self, *args, **kwargs): sup = lambda: super(CachingQuerySet, self). get(*args, **kwargs) if len(args) != 1 or not kwargs or self.query.where: return sup() key, value = kwargs.iteritems().next() if key.endswith("__exact"): key = key[:-len("__exact")] if key not in ["pk", self.model._meta.pk.name]: return sup() cache_key = "%s:%s:%s" % ( self.model._meta.app_label, self.model._meta.object_name, value ) obj = cache.get(cache_key) if obj is not None: return obj obj = sup() cache.set(cache_key, obj) return obj
  • from django.db.models.signals import pre_save, pre_delete class CachingManager(QuerySet): use_for_related_fields = True def get_query_set(self): return CachingQuerySet(self.model) def contribute_to_class(self, *args, **kwargs): super(CachingManager, self). contribute_to_class(*args, **kwargs) pre_save.connect(invalidate_cache, self.model) pre_delete.connect(invalidate_cache, self.model)
  • def invalide_cache(instance, sender, **kwargs): cache_key = "%s:%s:%s" % ( instance._meta.app_label, instance._meta.object_name, instance.pk ) cache.delete(cache_key) Aaand, Done!
  • Questions?
  • Extra Custom Field Review
  • Methods Converts value from to_python serialized form to Python validate Performs validation db_type Returns the database type Performs DB agnostic get_prep_value coercion and validation DB specific coercion/ get_db_prep_value validation formfield Provides a form field