Building open source with django


Published on

Ed Crewe of Django Bath & Bristol Users Group, presented at the Bristol IT MegaMeet 2013

Django is a leading web framework, in Python, used by the likes of Mozilla, Instagram and OpenStack. BOS is a survey system developed by the R&D team over the last decade, and in use by most UK Universities.

Find out how we are rewriting BOS from scratch in Django, Cassandra and Postgres, using current agile practises. Along with how to package a component of a larger product, to contribute back to an open source tool set.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Building open source with django

  1. 1. Building open source with Django Ed Crewe #dbbug Django Bath and Bristol Users Group
  2. 2. Ed Crewe * - DBBUG Python and Django developer. Representing Django Bristol and Bath User Group. Django is a leading web framework, in Python, used by the likes of Mozilla, Instagram and OpenStack. It is the framework bundled with Google App Engine - whose team is lead by the inventor of Python, Guido Van Rossum. Introduction * Developer at the University of Bristol (& amateur painter - see slide pics!) - I used to work at UWE before that!
  3. 3. Python is a high level programming language, it inherits many of the same principles set out by the first programming language explicitly designed to be object orientated, Smalltalk (1970) Python was invented at the end of the 1980s and a number of its concepts were shared by Java when it came along a few years later. The main difference being Java's static typing and verbose syntax, rather than Python's / Smalltalk's strong dynamic typing. The core areas of Python use are scientific / numerical analysis (replacing Fortran), educational tools (Raspberry Pi etc.), config management, and of course web software. What is python?
  4. 4. The Zen of Python The key ethos of Python is quite well adhered to by the creators of Django. Simple, explicit, DRY (inheritance), less code, RAD, loose coupling tight cohesion. Python lets you do what you want - procedural or OOP, multiple inheritance or interfaces, pre-compiled or meta / interactive programming ... so unusually diverse in methodology terms, but it has very strict rules of convention - break them and you advertise your failings - but those failings will still compile. There should ideally only be one syntactic way any line of code is written, and that way is the simplest and most readable way - see the PEPs (Python Enhancement Proposals). However a core principle of Python coding is also 'It is easier to ask forgiveness, than permission' coined by Grace Hopper Hence exception handling for program flow is OK, programming methodologies or typing should not be enforced - so you can go do what you want, as long as you write for readability, then ask forgiveness ... if it doesn't pass PEP lint tests!
  5. 5. Django was created around the same time as Ruby on the Rails and open sourced a few months afterwards in 2005. Like Rails it is MVC based and one of the top ten most popular web frameworks, alongside ASP MVC, CakePHP, Spring, Flask etc. Its a medium complexity orientated framework, sitting between Spring and the micro-frameworks (eg. Flask) on the scale of large development vs. productivity. So for systems that require less than 10 full time developers - but are more than a few weeks work for one. Like most significant open source projects it has a non-profit foundation to promote it and protect its licences. Probably the most well known example is the Apache foundation* - which looks after many smaller open source projects, particularly Java based ones, perhaps because due to its commercial heritage Java doesn't have its own language foundation like Python, Perl etc. What is django? * Colleague Damian Steer in Bristol Uni ITS R&D is a member
  6. 6. Started by Dan Fairs in 2008 ~ 100 members. Mainly developers with a scattering of project managers. The Django Bath and Bristol Users Group is a place for local developers and companies to come and talk Django, Python, and web development in general. We run events every couple of months, with a mix of socials and technical meets. #dbbug Django Bath and Bristol Users Group Membership is a mix of freelancers and company members, eg. from The House of Omni, ISM Games, Potato, SecondSync, Digital TV labs, Maplecroft, Joinerysoft, Netsight, Bravo6 etc. plus the odd public sector bod like me from the University of Bristol and Cardiff University
  7. 7. #dbbug What do we do? So it may be lightning talks or more a more in depth talks from a member. We tend to alternate, talk sessions with socials. Even at the talk sessions there will usually be some social lubrication such as beer and pizza - hopefully paid for by a recruitment company! We have also done a django dash (a competitive sprint), and a group outing to, a couple of years ago when it was in Amsterdam.
  8. 8.!forum/dbug Please come along to our next meet. Not that we have a date for it yet, but just email the list at Google group search for dbbug ... if you email and ask to meet, it will prompt somebody to suggest a date for the next one - so you can be the inspiration for that gathering! You don't have to be a developer or a Python user already, but it helps if you aim to be one. List is mainly for group events (sometimes gets job offers and newbie questions too) #dbbug Come and join us ( don't have to be an evangelist)
  9. 9. Django is being used at the University to provide an integration framework for web applications that need to appear on our public web. So this integrates with our institutional CMS driven content and central data sources (e.g. We are also using it to rewrite our national survey system BOS (like survey monkey for academic and cross institutional provision) Django use at Bristol University
  10. 10. Background - the survey application
  11. 11. 1. Daily scrum to report progress and decide current work units 2. Sprints of 3-4 weeks duration 3. Back log to handle sprint over spill 4. Product owner to help lead overall work prioritisation 5. Continuous Integration - nightly builds for various test suites 6. Maybe not full TDD but good test coverage 7. Continuous refactor effort Agile practises B.C.S. SCRUM talk by Nigel Baker - Agile Bear
  12. 12. Splitting a large development project into packages Caters for separate tests, documentation, release management and development cycles. Encourages increase in modularisation and encapsulation - helps to decouple the code. (Macro level OOP!) Adding generic functionality - separate package can be used across projects. Aim for it to be entirely generic then it can be released - this then opens it up to being freely tested and improved and / or influence other components of an open framework Packaging functional components
  13. 13. In python (and hence django) packages are called eggs. There is a central repository of all open source eggs called PyPI (the cheese shop), like CPAN. PyPI is built into the core language packaging features. Create a folder add a few files including, to make it an egg, and run python register to push it to PyPI. A user of your code then just has to download and run python setup or use the pip add on and run pip install package-name to add it to their django based site. Python packages
  14. 14. Django requires cookies to maintain session, and hence for authorisation. However there are cases when forms are used on public sites, where setting cookies is not desirable (due to privacy legislation), since technically they are not required for anonymous users to respond to forms. So if used, strictly consent is needed from the user to set them. This package is designed to cater for anonymous user session maintenance, without cookies. It provides a sessions implementation and decorator class for views to allow for forms to maintain state without using cookies by posting the encrypted session id between forms, or via urls. This also enables a user to run multiple sessions across different browser tabs. e.g. django-cookieless
  15. 15. Open source ethos Python ethos = OS ethos - code should be readable, openly accessible, reusable and adaptable by everyone. So open source should be well documented and maintained with test suites, release management. To deliver these goals. Note that the majority of commercial software is also now, at least partially, open source.
  16. 16. Sharing code To start sharing code, just reply to somebody with some. Maybe on a platform email list or stack overflow or the like. If its a longer snippet you may want to use your platform's favourite paste bin site, eg. to share it. If you think its of general use why not make it more permanently public, for django that means If its more than a snippet then it should be in a versioning system, so why not pick a public one ...
  17. 17. Like Facebook, but with a purpose! Community coding Site Pros Cons market leader only does git (Atlassian) git or mercurial * free private repos Its written in django :-) has been accused of plaigarising github git, mercurial or svn (for the old folks) poorly featured, few signs of new releases like github but open source infrastructure only does git bit flakey and smaller scale the original ... ... and worst. graveyard of old projects * Mercurial is the Python twin of Git (written in C), separated at birth!
  18. 18. 1. Reply to help lists - stack overflow or python and django specific 2. Reply to project support chat rooms 3. Reporting bugs 4. Fixing bugs 5. Adding new features to existing packages 6. Contributing enough to warrant a new release 7. Coagulating snippets, paste bins, stack overflows into an egg 8. Creating and releasing your own package 9. Acting as a release manager 10. Becoming a core contributor of a large project 11. Starting your own large project / framework / language (Warning 11 is not for people who have a life outside coding) The open road
  19. 19. Creative co-operation Githubs fun visualisation of contributors to one of my eggs .... hmm need to catch up merging those forks, or I will lose release manager / origin status for it ...
  20. 20. Don't be afraid to commit Contributing to Django core, or other Python projects (Boosting you half way along the open road!) A workshop next week, 8th June, by fellow DBBUG member, Daniele Procida (Details - room 3 - 1:50pm) See for more information. The workshop will take participants through the complete cycle of identifying a simple issue in a Django or Python project, writing a patch with tests and documentation, and submitting it. The workshop is aimed at the first-time committer. Very little experience is required: https://dont-be-afraid-to-commit.readthedocs. org/en/latest/prerequisites.html
  21. 21. Interested in Django ... home dbbug!forum/dbug free book 90 other books keywords=django+python chat #django IRC help!forum/django-users core devs!forum/django-developers other dbbugs jobs and add ons
  22. 22. Ed Crewe!forum/dbug #dbbug django bath and bristol user group university of bristol - ITS - R&D Contact PS: Also a member of SW Big Data /south-west-big-data/