Building a custom cms with django


Published on

Presentation given at 2010-08-29

Published in: Technology
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Building a custom cms with django

  1. 1. Building a custom CMS Process to build a CMS with django Brian Luft <> Yann Malet <> – 27 aout 2010 – Paris
  2. 2. Agenda for the presentation 1. Planning and Methodology 2. Dealing With Legacy Data Stores 3. Building it
  3. 3. Planning and Methodology Digesting, Evaluating, and Structuring the Work Ahead 1.Planning and Methodology 2.Dealing With Legacy Data Stores 3.Building it
  4. 4. Migrating existing content vs building the new CMS These 2 tasks are antagonist and you will need to constantly context switch to make progress on both front Migrating existing content: • Helps you understand the goal of your customer • Gives you "real data" to evaluate your development • Most precious asset of your customer • Hard and time consuming Building the new CMS • You are evaluated and paid for this • Fun and creative part of the job • Opportunity to improve content work flows
  5. 5. Methodology Prototype driven development based on a succession of 2 weeks long sprint Typical project phases: • Create a prototype as fast as possible • Populate it with "real data" as soon as you can • Load test your prototype to find the bottlenecks • ...and iterate until you are done
  6. 6. Methodology Iterate on this virtuous cycle: • develop • QA • demo Developing in iterations helps keep project metrics in check (you know, the things that always get abandoned): • Code quality • Documentation • Unit tests and test coverage At the conclusion of each phase you should have new working features and code you're proud of.
  7. 7. Team Communication Keep stakeholders involved and aware as much as possible. Enforce regular meetings PM should make sure tickets, questions get answered. Tools : • IRC room per project • Sprint Demo • Integration server (WIP) • Backlog monitoring (Redmine & co) • Forget Wikis, Maintain good project (Sphinx) docs
  8. 8. Key success factors • First you build a raw estimate o Experience helps o Even still, things always take longer. Pad, pad, pad. • Continuously refine your estimate • Demo finished features on a fix schedule o Keep your build working o Keep stakeholders in the loop o Define an escalation process • Pin down early on what the most complex aspects will be. Make sure you are chipping away at them rather than pushing everything back until the end.
  9. 9. Dealing With the Legacy Systems 1.Planning and Methodology 2.Dealing With Legacy Data Stores 3.Building it
  10. 10. Migrating Legacy Data Two opposing factors to consider: 1. The "real" migration will only ever happen one time. – The data is the business - it has to be handled properly. Plan ahead: • How are you going to keep migration plan in sync with changes in your new application? • Don't underestimate the work required to have a workable process • The modify-migrate-test cycle is slow.
  11. 11. Migrating the Data A convenient way to migrate your data is very important because you will need to run it often. No matter how much time you spend to optimize this. • it will always not be fast enough Optimize the ease of run rather than the speed... • put it in the cloud • or another computer ... because your new data model will evolve and you will need to run this often
  12. 12. Migration Pitfalls You'll probably encounter many of these problems: • Other frameworks and databases use different conventions or even other data types for FK then standard Django • Mapping to Django User is problematic • Any large dataset is inconsistent • Fix the integrity at the source when possible • Naive migration strategy will leave you a process that takes many hours or even days.
  13. 13. Migration Pitfalls continued • Your new models will go through many evolutions as you continually adapt various pieces of complex logic (and hacks) into a more elegant schema. • Gracefully handling stuff in legacy content fields (HTML tags/snippets, entities, encoded chars)
  14. 14. Migration Tips • Dataset o Visual inspection of the dataset o Gather as much legacy app logic and SQL as you can up front. Working backwards through relationships is tedious • Django specific o Use inspectdb to navigate the original dataset o Use multidb to move the data from the old schema o Don't burn time fighting the ORM. Use the cursor with raw SQL or reach for other Python tools.
  15. 15. Features for your migration tool set Sanity must haves for your tool(s): • Pause / Clean break • Resume • Progress meter • Logging • Profiling • Partial / Range • Graceful Error Handling
  16. 16. Building It How to Decide What to Use and What to Create 1.Planning and Methodology 2.Dealing With Legacy Data Stores 3.Building it
  17. 17. Pitfalls -- Create your own wonderland It is easy to create a system that is completely undjango-ish due to external constraints and influence from legacy system. • There is often a temptation for developers new to Django to structure code or use idioms from other frameworks/languages. • Trying to "clone" a large legacy system might lead you astray of commonly accepted Django best practices. • Possible consequences: o Reusable apps won't plug in well o Updates to Django trunk might not work o New/external developers might be lost / need training o Integrating existing Django project is HARD
  18. 18. Customize or Build Your Own? You'll have to evaluate on many levels whether using 3rd party code is a good idea: • Starting with application templates as your foundation o django-startproject o pinax • Using 3rd party apps for pieces of your site o django-cms o satchmo o ... •
  19. 19. Customize or Build Your Own? continued Our philosophy is to favor 3rd party components but not be afraid to fork early. Pick out the best parts and use those. • use your network of trust to evaluate an app • check up the code quality metrics o test coverage o complexity o documentation o author / number of followers Shoehorn "out-of-the-box" on large project might cause you more hassle than good • customized • extend
  20. 20. Engage the 3rd party app community This community is composed of the most knowledgeable people in this realm. A good connection will enable you to : • Fix bugs: get patches in trunk / master • influence the evolution by suggesting o new feature o enhancement of the existing code base • understand existing design decisions • get free, instant, expert advice
  21. 21. How to effectively contribute to a project Embrace the project • communication channel (IRC, Bug tracker, ...) • infrastructure (dvcs, test suite, ...) Run the extra miles that will make the difference • When you report a bug o create a test that illustrate it • if you provide a patch o Make sure you provide it in the most adequate format for the core developer Help them to help you...
  22. 22. Get your contribution "IN" You have worked on : • a bug report • attach a test • provided a patch that fixes the issue • written the documentation Which key action is missing to get it in ? • Engage the core developers in a continuous exchange o This issue must be on their radar and they should be aware of your progress
  23. 23. Conclusion • These are very exiting projects • django with its ecosystem is well suited for the job • Get help from external an external company • Fixed price contract is almost impossible because nobody knows the exact scope
  24. 24. Time for questions ... Thank Yann Malet <>