Dynamic Models
in Django
Juergen Schackmann
j.schackmann@reliatec.de
© Juergen Schackmann 2013 2/28
Me
▸ Python & Django since 4 years
▸ reliatec.de since 3 months
▸ Django Web Applications
▸ Focus: EDC systems for clinical studies & Telemonitoring
© Juergen Schackmann 2013 3/28
Objective
▸ Why do do we care about Django Dynamic Models?
▸ Why do we have to talk about them?
▸ What should we do about them?
Static vs Dynamic Models
© Juergen Schackmann 2013 5/28
Models in a Django project
Phase1:
Code Development
Models and model
attributes are defined by
implementing those with
Python code
Phase 2:
Deployment
Underlying database tables
and columns are created by
running “syncdb” command
Phase 3:
Runtime
Models / DB Tables are
populated with content as
part of the user
interactions
© Juergen Schackmann 2013 6/28
Problem?
There is no way to adapt the Django Models at runtime based on
user interactions
▸ Django Models are defined and implemented during
development phase
▸ Django Models are expected to be static after deployment
▸ Django DB Backends only fully support SQL databases, i.e. a
schema is required
▸ DB specific solutions limit adoption rate of any reusable App
Use Cases for Dynamic
Models
© Juergen Schackmann 2013 8/28
Custom Content Types for a CMS*
* system: dotCms (Java); source: http://dotcms.com/
© Juergen Schackmann 2013 9/28
Customized Products for an Ecommerce Shop*
* system: Magento (PHP); source: averybrandandprint.co.uk
© Juergen Schackmann 2013 10/28
Clinical Research Forms*
* system: OpenClinica (Java); source: openclinica.com
© Juergen Schackmann 2013 11/28
Problem? Which Problem?
▸ Models / Content Types are domain or site specific
▸ Neither realistic nor efficient to implement all thinkable content
types into a software product
▸ Configuration by subject-matter experts is better, than code
customization by developers
▸ Dynamic Models reduce the number of deployment cycles
Evaluation of
Patterns & Apps
© Juergen Schackmann 2013 13/28
As a User, I ...
▸ want to maintain my dynamic models easily without
understanding any Django Model details
▸ want extend existing (static) models without thinking about
static and dynamic parts
▸ do not want to think about performance, when creating my
custom fields
▸ I want to be able to query/search for certain information
seamlessly over static and dynamic models
© Juergen Schackmann 2013 14/28
As a developer, I ...
▸ want my dynamic models to be integrated with Django Standard
Tools (admin, migrations, reversion) out of the box
▸ want to apply the same API that I use for the static models
▸ do not want to use or be limited to specific databases, but use
the standard Django backends
▸ want to have a customizable front-end for my users to manage
the dynamic models
© Juergen Schackmann 2013 15/28
Evaluation Criteria
▸ Performance
▸ “Querybility”
▸ Django Standard Tools Integration (Admin, Cache, ...)
▸ Supported Backend
▸ Complexity/ Maintainability
© Juergen Schackmann 2013 16/28
Entity Attribute Value (EAV)
Transform your table columns into table rows
Attribute 1 Attribute 2 Attribute n
Table
Attribute
Name
Attribute
Value
Attribute
TableTable has1 n
© Juergen Schackmann 2013 17/28
EAV Django Apps
▸ bitbucket.org/neithere/eav-django/
▸ github.com/mvpdev/django-eav
© Juergen Schackmann 2013 18/28
Serialized Dictionary
Transform your table columns into pickeld or jsoned dictionaries
Attribute 1 Attribute 2 Attribute n
Table
Dictionary
String
Attributes
{“attribute1”:”a”, “attribute2”: 5, “attribute3”:[1,2] }
© Juergen Schackmann 2013 19/28
Serialized Dictionary Django Apps
Fields
▸ bitbucket.org/schinckel/django-jsonfield/
▸ github.com/bradjasper/django-jsonfield
▸ github.com/derek-schaefer/django-json-field
▸ github.com/shrubberysoft/django-picklefield
App
▸ github.com/dobarkod/django-dynamic-model
© Juergen Schackmann 2013 20/28
Runtime Schema Updates
Update Django models and database schema at runtime using
▸ syncdb functionality
▸ South functionality
▸ eventually: Andrew Godwin's “Schema Migrations for Django”
© Juergen Schackmann 2013 21/28
Runtime Schema Update Django Apps
Update Django models and database schema at runtime using
▸ Syncdb
▸ bitbucket.org/mhall119/dynamo/overview
▸ South
▸ github.com/willhardy/dynamic-models
▸ github.com/willhardy/django-dymo
▸ bitbucket.org/schacki/django-dynamo
▸ github.com/charettes/django-mutant
▸ Schema Migrations
▸ github.com/charettes/django-mutant
© Juergen Schackmann 2013 22/28
Database specific solutions
Hstore
▸ github.com/jordanm/django-hstore
▸ postgresql.org/docs/9.0/static/hstore.html
No SQL
▸ github.com/django-nonrel
▸ github.com/pydanny/django-mongonaut
© Juergen Schackmann 2013 23/28
Summary
EAV Serialization Schema Updates
Performance   
Querybility   
Django Standard
Tools
 / 
Supported
Backend
  
Complexity/
Maintainability
  
© Juergen Schackmann 2013 24/28
Schema Update most powerful, but ...
▸ Name clashes between developer models and dynamic models
(especially with schema migrations)
▸ Multiple Users updating the same model (possibly even
conflicting)
▸ Orphan database elements
▸ Dynamic Model loading at startup
▸ How to find/inject dynamic models into views and templates
Components of a
Dynamic Model App
© Juergen Schackmann 2013 26/28
Must have components of a Dynamic Model
Application
▸ Meta Model Front End to let users maintain their meta models
▸ Meta Model Representation to store the meta model information
▸ Meta Model Analyzer to detect changes that require some further
actions (for schema, admin or model)
▸ Schema Sync Manager to update and sync the database schema
▸ Admin Sync Manager to update and sync the Django admin
▸ Model/Cache Sync Manager to update and sync the Django models
and model cache
© Juergen Schackmann 2013 27/28
More resources on dynamic models
▸ code.djangoproject.com/wiki/DynamicModels
▸ djangopackages.com/grids/g/forms/
▸ stackoverflow.com/questions/7933596/django-dynamic-model-
fields/7934577
Thank you very much!
Questions please?

Dynamic Models with Django

  • 1.
    Dynamic Models in Django JuergenSchackmann j.schackmann@reliatec.de
  • 2.
    © Juergen Schackmann2013 2/28 Me ▸ Python & Django since 4 years ▸ reliatec.de since 3 months ▸ Django Web Applications ▸ Focus: EDC systems for clinical studies & Telemonitoring
  • 3.
    © Juergen Schackmann2013 3/28 Objective ▸ Why do do we care about Django Dynamic Models? ▸ Why do we have to talk about them? ▸ What should we do about them?
  • 4.
  • 5.
    © Juergen Schackmann2013 5/28 Models in a Django project Phase1: Code Development Models and model attributes are defined by implementing those with Python code Phase 2: Deployment Underlying database tables and columns are created by running “syncdb” command Phase 3: Runtime Models / DB Tables are populated with content as part of the user interactions
  • 6.
    © Juergen Schackmann2013 6/28 Problem? There is no way to adapt the Django Models at runtime based on user interactions ▸ Django Models are defined and implemented during development phase ▸ Django Models are expected to be static after deployment ▸ Django DB Backends only fully support SQL databases, i.e. a schema is required ▸ DB specific solutions limit adoption rate of any reusable App
  • 7.
    Use Cases forDynamic Models
  • 8.
    © Juergen Schackmann2013 8/28 Custom Content Types for a CMS* * system: dotCms (Java); source: http://dotcms.com/
  • 9.
    © Juergen Schackmann2013 9/28 Customized Products for an Ecommerce Shop* * system: Magento (PHP); source: averybrandandprint.co.uk
  • 10.
    © Juergen Schackmann2013 10/28 Clinical Research Forms* * system: OpenClinica (Java); source: openclinica.com
  • 11.
    © Juergen Schackmann2013 11/28 Problem? Which Problem? ▸ Models / Content Types are domain or site specific ▸ Neither realistic nor efficient to implement all thinkable content types into a software product ▸ Configuration by subject-matter experts is better, than code customization by developers ▸ Dynamic Models reduce the number of deployment cycles
  • 12.
  • 13.
    © Juergen Schackmann2013 13/28 As a User, I ... ▸ want to maintain my dynamic models easily without understanding any Django Model details ▸ want extend existing (static) models without thinking about static and dynamic parts ▸ do not want to think about performance, when creating my custom fields ▸ I want to be able to query/search for certain information seamlessly over static and dynamic models
  • 14.
    © Juergen Schackmann2013 14/28 As a developer, I ... ▸ want my dynamic models to be integrated with Django Standard Tools (admin, migrations, reversion) out of the box ▸ want to apply the same API that I use for the static models ▸ do not want to use or be limited to specific databases, but use the standard Django backends ▸ want to have a customizable front-end for my users to manage the dynamic models
  • 15.
    © Juergen Schackmann2013 15/28 Evaluation Criteria ▸ Performance ▸ “Querybility” ▸ Django Standard Tools Integration (Admin, Cache, ...) ▸ Supported Backend ▸ Complexity/ Maintainability
  • 16.
    © Juergen Schackmann2013 16/28 Entity Attribute Value (EAV) Transform your table columns into table rows Attribute 1 Attribute 2 Attribute n Table Attribute Name Attribute Value Attribute TableTable has1 n
  • 17.
    © Juergen Schackmann2013 17/28 EAV Django Apps ▸ bitbucket.org/neithere/eav-django/ ▸ github.com/mvpdev/django-eav
  • 18.
    © Juergen Schackmann2013 18/28 Serialized Dictionary Transform your table columns into pickeld or jsoned dictionaries Attribute 1 Attribute 2 Attribute n Table Dictionary String Attributes {“attribute1”:”a”, “attribute2”: 5, “attribute3”:[1,2] }
  • 19.
    © Juergen Schackmann2013 19/28 Serialized Dictionary Django Apps Fields ▸ bitbucket.org/schinckel/django-jsonfield/ ▸ github.com/bradjasper/django-jsonfield ▸ github.com/derek-schaefer/django-json-field ▸ github.com/shrubberysoft/django-picklefield App ▸ github.com/dobarkod/django-dynamic-model
  • 20.
    © Juergen Schackmann2013 20/28 Runtime Schema Updates Update Django models and database schema at runtime using ▸ syncdb functionality ▸ South functionality ▸ eventually: Andrew Godwin's “Schema Migrations for Django”
  • 21.
    © Juergen Schackmann2013 21/28 Runtime Schema Update Django Apps Update Django models and database schema at runtime using ▸ Syncdb ▸ bitbucket.org/mhall119/dynamo/overview ▸ South ▸ github.com/willhardy/dynamic-models ▸ github.com/willhardy/django-dymo ▸ bitbucket.org/schacki/django-dynamo ▸ github.com/charettes/django-mutant ▸ Schema Migrations ▸ github.com/charettes/django-mutant
  • 22.
    © Juergen Schackmann2013 22/28 Database specific solutions Hstore ▸ github.com/jordanm/django-hstore ▸ postgresql.org/docs/9.0/static/hstore.html No SQL ▸ github.com/django-nonrel ▸ github.com/pydanny/django-mongonaut
  • 23.
    © Juergen Schackmann2013 23/28 Summary EAV Serialization Schema Updates Performance    Querybility    Django Standard Tools  /  Supported Backend    Complexity/ Maintainability   
  • 24.
    © Juergen Schackmann2013 24/28 Schema Update most powerful, but ... ▸ Name clashes between developer models and dynamic models (especially with schema migrations) ▸ Multiple Users updating the same model (possibly even conflicting) ▸ Orphan database elements ▸ Dynamic Model loading at startup ▸ How to find/inject dynamic models into views and templates
  • 25.
  • 26.
    © Juergen Schackmann2013 26/28 Must have components of a Dynamic Model Application ▸ Meta Model Front End to let users maintain their meta models ▸ Meta Model Representation to store the meta model information ▸ Meta Model Analyzer to detect changes that require some further actions (for schema, admin or model) ▸ Schema Sync Manager to update and sync the database schema ▸ Admin Sync Manager to update and sync the Django admin ▸ Model/Cache Sync Manager to update and sync the Django models and model cache
  • 27.
    © Juergen Schackmann2013 27/28 More resources on dynamic models ▸ code.djangoproject.com/wiki/DynamicModels ▸ djangopackages.com/grids/g/forms/ ▸ stackoverflow.com/questions/7933596/django-dynamic-model- fields/7934577
  • 28.
    Thank you verymuch! Questions please?