The document discusses strategies for scaling Django applications, including optimizing queries, schema design, caching, load balancing, database partitioning, and sharding. It provides examples of best practices for modeling data, debugging queries, and architectural approaches for scaling the database. Key recommendations include using appropriate field types, adding indexes strategically, and approaches that keep the application logic separated from database scaling.

1. Pulling more out of Django application Supreet Sethi Email < [email_address] This talk is possible because of kindness of some clients, who let me use their machinery for my utterly wierd experiments

2. Perspective on databases Strictly structured data Sophisticated relation description language Depends heavily on locality of data and indexing

3. Perspective on databases 2 ORM see database as storage ORMs are less sophisticated in mapping relations 1-1 mapping is sometimes not possible Atomicity of operations cannot of insured

4. Development cycle with databases Migraine Designing schema Programming business logic Adding fixtures

5. With Django Code on diet Model Templates View Model serves as one point place for data schema and business logic Url patterns

6. Understanding Django application stack Server Database Layer User Interface Business Logic Common Storage

7. Problem with the stack Tightly coupled

8. General overview of django scaling Using javascript to load as much dynamic data making static content in pages cacheable (AJAX) Compress applications CSS and javascripts (django-compress) Optimize pagination Connection pooling Memory Caching

9. Database scaling for django application Schema design and modeling Debug queries Data specific changes in the model Database architecture for scaling

10. Schema design and modeling Using appropriate field type CharField for small text DateField for date and time Text, Integer, Slug, Email, Url, Image, File Using hinting whereever possible or required Define primary_key for primary key field Map relations well Use choice modifier BooleanField Unique modifier

11. Class Pizza(models.Model): pizza_type = models.CharField(choices=(('ch','chicken'), ('pr':'pepperoni'),) crust_type = models.CharField(choices=(('hr','hard'), ('fil', 'filled'),) topping_type = models.CharField(choices=(('fr', 'farm fresh veggies', ('hu','hunk of meat'),) date_of_order = models.DateTimeField(primary_key=true) Class Pizza(models.Model): pizza_type = models.CharField() crust_type = models.CharField() topping_type = models.CharField() date_of_order = models.CharField() Wrong Right

12. Debug the queries http://github.com/robhudson/django-debug-toolbar/tree/master

13. Data specific changes in the model db_index modifier on fields Be sure to use them carefully Increases the write costs on the field Helps in seeking data faster Unique constraint with unique modifier Recheck relations between tables

14. Database architecture for scaling (1) Loadbalancing Application is unaware of the scaling logic Database can be loaded using django admin Normalization of related tables is not required Its scalable linearly Orders 2007 Load balancer Application Server 3 Orders 2007 Orders 2007 Orders 2007 Application Server 1 Application Server 2

15. Database architecture for scaling (2) Partitioning Transparent to django application Database cannot be loaded using django admin Normalization of related tables does not change Its scalable linearly upto database limits Pizza orders database Order 2007 Orders 2009 Orders 2008 Application Layer

16. Database architecture for scaling (3) Sharding Application should be aware of the sharding logic Database cannot be loaded using django admin Normalization of related tables is required Its scalable linearly Orders 2007 Application Layer Sharding logic Orders 2007 Orders 2007 Orders 2007

17. Example of sharding http://www.eflorenzano.com/blog/tag/database-sharding/

18. Where not to use django Where atomicity of operation is required Where data needs to be updated on tables and pulled using a view Complex schemas