State of
 James Tauber


                DjangoCon 2009
pi in hex
def pi():
    N = 0
    n, d = 0, 1
    while True:
        xn = (120*N**2 + 151*N + 47)
        xd = (512*N**4 + 1024*N**3 +
              712*N**2 + 194*N + 15)
        n = ((16 * n * xd) + (xn * d)) % (d * xd)
        d *= xd
        yield 16 * n // d
        N += 1
Your Site
Your Site
            Django



 Python      Python
idea   realisation
Quisition
            • currently has:
              • signup, login, password
                management
              • announcements
              • deck searching and sort

            • want to add:
              • user profiles
              • rating, tagging and commenting
                  on decks
              •   groups, friendships
              •   membership levels
              •   notifications
              •   OpenID
Hot Club of France

      reusable apps

     app conventions
Potter Predictions
               • send invitations
               • connect to friends
               • make comments on
                 predictions
               • tag prediction
               • mass mailout to
                 users
Tabula Rasa
          reusable apps

         app conventions


                +
  starting point for new projects

scaffolding for developing new apps
If you want to build a ship, don’t drum up the men to
        gather wood, divide the work and give orders.
Instead, teach them to yearn for the vast and endless sea.

                                 – Antoine de Saint-Exupéry
Let’s Build a
Social Networking Site
{ n | available(“cloud<n>.com”), n < 30 }
                      =
              { 14, 26, 27, 28 }
collection of reusable apps

  conventions for reusable apps

  starting point for new projects

scaffolding for developing new apps

               +
a platform for website development
Your Project
Your Project
                  Pinax



  Django         Django




  Python         Python
•   AuthX (AuthN + AuthZ)

Modern Web Infrastructure                                       ◦ Capabilities
                                                                ◦ Multifactor Auth
                                                                ◦ Rate Limiting
                                                                ◦ Signup
                                                                ◦ OpenID
                                                                ◦ OAuth
                                                                ◦ External import
   •   API Metering                                       •   Groups
   •   Backups & Snapshots                                •   Invites
   •   Counters                                           •   Lists
   •   Cloud/Cluster Management Tools                     •   Notifications
         ◦ Instrumentation/Monitoring (Ganglia, Nagios)         ◦ Spam filtering
         ◦ Failover                                             ◦ Multi-protocol routing
         ◦ Node addition/removal and hashing                    ◦ Fine-grained controls/rules
         ◦ Autoscaling for cloud resources                •   Presence
   •   CSRF/XSS Protection                                •   Social Activity Log (Newsfeed)
   •   Data Retention/Archival                                  ◦ Filtering
   •   Deployment Tools                                   •   Social Model
         ◦ Multiple Devs, Staging, Prod                         ◦ Connectivity (uni/bidi)
         ◦ Data model upgrades                                  ◦ Privacy (private, reciprocal, public)
         ◦ Rolling deployments                                  ◦ Views
         ◦ Multiple versions (selective beta)                   ◦ Traversal
         ◦ Bucket Testing                                 •   Social Object
         ◦ Rollbacks                                            ◦ Privacy, Social Scoping
         ◦ CDN Management                                       ◦ Voting
   •   Distributed File Storage                                 ◦ Sharing
   •   Distributed Log storage, analysis                        ◦ Publishing
   •   Graphing                                                 ◦ Comments
   •   HTTP Caching                                             ◦ Favoriting
   •   Input/Output Filtering                                   ◦ Social editing
   •   Memory Caching                                           ◦ Permissions
   •   Non-relational Key Stores                          •   Tagging
   •   Rate Limiting                                            ◦ Combinations
   •   Relational Storage                                       ◦ Relatedness
   •   Queues                                             •   User
   •   Rate Limiting                                            ◦ Achievements/Awards
   •   Real-time messaging (XMPP)                               ◦ Activity Log
   •   Search                                                   ◦ External User ID Mapping
         ◦ Ranging                                              ◦ Permissions (see AuthX)
         ◦ Geo                                                  ◦ Deletion/Archival
   •   Sharding                                                 ◦ Flagging
   •   Smart Caching                                            ◦ Direct Messaging
         ◦ dirty-table management                               ◦ User Cards
What seems the biggest shame to me is that everyone is
currently rebuilding this stuff over and over again and rationalizing
it as some sort of secret sauce competitive advantage when it’s
really infrastructure - stuff that really should be standardized so
you can actually get around to doing the new and interesting
stuff.
                                                                      — Leonard Lin
                              http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
Your Project
               the trade off:
   Pinax
               how much to fix?
               how much to make configurable?
               how much to leave
  Django
               completely to site developer?
Reusability is not just
  at the app layer
reusable app




quisition          potterpredictions
djangopeople   pythonpeople
pinax project




djangopeople             pythonpeople
Ole Kirk Christiansen
• blocks
• sets
• themes
Pinax IS Django
Pinax IS NOT Django
“Pinax is
opinionated so
Django doesn’t
 have to be”
“PINAX:
   Even more
opinionated than
    Django”
Milestones

  •   0.5.0 — October 2008
  •   0.5.1 — January 2009
  •   0.7beta1— April 2009 (485)
  •   0.7beta2 — April 2009 (58)
  •   0.7beta3 — July 2009 (445)
  •   0.7rc1— September 2009 (291)
1271 commits
343 tickets closed
 between 0.5 and 0.7
 (not counting external apps)
Features in 0.5
•   openid support                                 • interest groups
                                                           (called tr ibes)
•   email ver ification
•   password management                            • projects
                                                        with basic task and issue management
•   site announcements                             • threaded discussions
•   a notification fr amewor k                     • wikis
•   user-to-user messaging                                 with multiple mar kup suppor t
•   fr iend invitation                             •   blogging
      (both inter nal and exter nal to the site)
                                                   •   bookmar ks
•   a basic twitter clone
                                                   •   tagging
•   oembed suppor t
                                                   •   contact impor t
•   gr avatar suppor t                                     (from vCard, Google or Yahoo)
                                                   • photo management
0.5 Structure
  • external apps
      • all on Google Code (SVN)
      • half written independently of Pinax
  •     local apps
  •     libraries (batteries included)
  •     two sample projects
  •     path hacking
Sites Using Pinax
Sites Using Pinax
Sites Using Pinax
Sites Using Pinax
Sites Using Pinax
0.7 Release
•   translation removal
•   locations
•   frontend-admin and generic flat-blocks
•   account and group improvements
•   new task app
•   uni-form and Section 508
•   better shared templates and media
•   more sample projects
    (and complete renamed social)

• virtualenv and pip
• pinax is now a package
New Account App

 •   turn OpenID on/off
 •   turn sign up on/off
 •   waiting list
 •   invite only / private beta
 •   token-based password reset
 •   (resend signup email confirmation)
 •   (use email for login)
you signed up for       admin initiated           each user has fixed        user can invite
       beta                account               number to hand out            anyone



           admin sent code                                        user invited you



    general code                          unique invite
                                                                       just go to site
      (type in)                            (email link)



         have code                                                     external auth




               username / password                 username
                     signup                      (no password)




                                  SIGNED UP
New Group App
 •   Don’t Repeat Yourself!
 •   endo vs exo
 •   group apps versus content apps
 •   membership app
Endo Approach
(framework-like)
Endo Approach
(framework-like)
Exo Approach
(library approach)
Exo Approach
(library approach)
Shared Templates and Media
 • look up:
  • project
  • pinax            new level (multiple themes)
  • app
 • media build command
 • ship with famfamfam silk icons
Problems with 0.5 structure
 • it largely relies on external dependencies
   being in svn and this is increasingly not the
   case (although it was when Pinax started)
 • it makes it difficult for Pinax itself to move
   away from svn
 • there is no management of dependencies
   between external dependencies, nor between
   particular projects in Pinax and their individual
   dependencies
virtualenv and pip
 • encourage externals apps to be distutils-
   compliant
 • pip for package install (at least during
   development)
 • optional “batteries included” release
 • virtualenv to provide isolation, remove
   need for path hacking and just because it
   kicks ass
release install
(download tarball, extract and cd)

$ python scripts/pinax-boot.py ../pinax-env
$ source ../pinax-env/bin/activate
(pinax-env)$ cd ..

(pinax-env)$ pinax-admin clone_project
                      social_project mysite
(pinax-env)$ cd mysite/
(pinax-env)$ python manage.py syncdb
(pinax-env)$ python manage.py runserver
development install
(download pinax-boot.py from github)

$ python pinax-boot.py --development ./pinax-env
$ source ./pinax-env/bin/activate
(pinax-env)$


(pinax-env)$ cd ./pinax-env/
(pinax-env)$ pip install --requirement
         src/pinax/requirements/external_apps.txt
Move to github
http://github.com/pinax/pinax/
35 contributors
108 members of pinax-core-dev list
    58 at PyCon

336 users on code.pinaxproject.com
403 watchers on github
    157 at PyCon

649 members of pinax-users list
    398 at PyCon
Pinax is NOT just for
   social networks
site
       site
              site
                     site
                              site
                                        site
                                               site
                                                      site
                                                             site
                                                                    site
                                                                           site
                                                                                  site
                                                  Learning
Social Networks             Intranets                                 e-commerce
                                                 Management


                                         Pinax

                                        Django
site
       site
              site
                     site
                              site
                                        site
                                               site
                                                      site
                                                             site
                                                                    site
                                                                           site
                                                                                  site
                                                  Learning
Social Networks             Intranets                                 e-commerce
                                                 Management


                                         Pinax

                                        Django
Editions

 •   social networking
 •   intranet
 •   learning management
 •   e-commerce
 •   clubs and associations
 •   conference management
 •   software project management
Roadmap
 • 0.7rc1 out last week
 • 0.7 final in next day or two
 • 0.8 “translation release”
     based off 0.7 and in parallel to 0.9
   •

 • 0.9 “big feature release” and invitations
     rethink friends: split into friends, contacts
   •
   •   rethink profiles
   •   workflow
   •   open stack
   •   calendar / events
   •   new apps
   •   new projects
   •   new editions?
Join Us During Sprint
(even if just in #pinax-sprint on freenode)
http://pinaxproject.com/

State of Pinax

  • 1.
    State of JamesTauber DjangoCon 2009
  • 3.
  • 4.
    def pi(): N = 0 n, d = 0, 1 while True: xn = (120*N**2 + 151*N + 47) xd = (512*N**4 + 1024*N**3 + 712*N**2 + 194*N + 15) n = ((16 * n * xd) + (xn * d)) % (d * xd) d *= xd yield 16 * n // d N += 1
  • 5.
    Your Site Your Site Django Python Python
  • 6.
    idea realisation
  • 7.
    Quisition • currently has: • signup, login, password management • announcements • deck searching and sort • want to add: • user profiles • rating, tagging and commenting on decks • groups, friendships • membership levels • notifications • OpenID
  • 8.
    Hot Club ofFrance reusable apps app conventions
  • 9.
    Potter Predictions • send invitations • connect to friends • make comments on predictions • tag prediction • mass mailout to users
  • 10.
    Tabula Rasa reusable apps app conventions + starting point for new projects scaffolding for developing new apps
  • 12.
    If you wantto build a ship, don’t drum up the men to gather wood, divide the work and give orders. Instead, teach them to yearn for the vast and endless sea. – Antoine de Saint-Exupéry
  • 13.
    Let’s Build a SocialNetworking Site
  • 16.
    { n |available(“cloud<n>.com”), n < 30 } = { 14, 26, 27, 28 }
  • 17.
    collection of reusableapps conventions for reusable apps starting point for new projects scaffolding for developing new apps + a platform for website development
  • 18.
    Your Project Your Project Pinax Django Django Python Python
  • 19.
    AuthX (AuthN + AuthZ) Modern Web Infrastructure ◦ Capabilities ◦ Multifactor Auth ◦ Rate Limiting ◦ Signup ◦ OpenID ◦ OAuth ◦ External import • API Metering • Groups • Backups & Snapshots • Invites • Counters • Lists • Cloud/Cluster Management Tools • Notifications ◦ Instrumentation/Monitoring (Ganglia, Nagios) ◦ Spam filtering ◦ Failover ◦ Multi-protocol routing ◦ Node addition/removal and hashing ◦ Fine-grained controls/rules ◦ Autoscaling for cloud resources • Presence • CSRF/XSS Protection • Social Activity Log (Newsfeed) • Data Retention/Archival ◦ Filtering • Deployment Tools • Social Model ◦ Multiple Devs, Staging, Prod ◦ Connectivity (uni/bidi) ◦ Data model upgrades ◦ Privacy (private, reciprocal, public) ◦ Rolling deployments ◦ Views ◦ Multiple versions (selective beta) ◦ Traversal ◦ Bucket Testing • Social Object ◦ Rollbacks ◦ Privacy, Social Scoping ◦ CDN Management ◦ Voting • Distributed File Storage ◦ Sharing • Distributed Log storage, analysis ◦ Publishing • Graphing ◦ Comments • HTTP Caching ◦ Favoriting • Input/Output Filtering ◦ Social editing • Memory Caching ◦ Permissions • Non-relational Key Stores • Tagging • Rate Limiting ◦ Combinations • Relational Storage ◦ Relatedness • Queues • User • Rate Limiting ◦ Achievements/Awards • Real-time messaging (XMPP) ◦ Activity Log • Search ◦ External User ID Mapping ◦ Ranging ◦ Permissions (see AuthX) ◦ Geo ◦ Deletion/Archival • Sharding ◦ Flagging • Smart Caching ◦ Direct Messaging ◦ dirty-table management ◦ User Cards
  • 20.
    What seems thebiggest shame to me is that everyone is currently rebuilding this stuff over and over again and rationalizing it as some sort of secret sauce competitive advantage when it’s really infrastructure - stuff that really should be standardized so you can actually get around to doing the new and interesting stuff. — Leonard Lin http://randomfoo.net/2009/01/28/infrastructure-for-modern-web-sites
  • 21.
    Your Project the trade off: Pinax how much to fix? how much to make configurable? how much to leave Django completely to site developer?
  • 22.
    Reusability is notjust at the app layer
  • 23.
    reusable app quisition potterpredictions
  • 24.
    djangopeople pythonpeople
  • 25.
  • 26.
  • 27.
  • 28.
  • 29.
  • 30.
    “Pinax is opinionated so Djangodoesn’t have to be”
  • 31.
    “PINAX: Even more opinionated than Django”
  • 32.
    Milestones • 0.5.0 — October 2008 • 0.5.1 — January 2009 • 0.7beta1— April 2009 (485) • 0.7beta2 — April 2009 (58) • 0.7beta3 — July 2009 (445) • 0.7rc1— September 2009 (291)
  • 33.
    1271 commits 343 ticketsclosed between 0.5 and 0.7 (not counting external apps)
  • 34.
    Features in 0.5 • openid support • interest groups (called tr ibes) • email ver ification • password management • projects with basic task and issue management • site announcements • threaded discussions • a notification fr amewor k • wikis • user-to-user messaging with multiple mar kup suppor t • fr iend invitation • blogging (both inter nal and exter nal to the site) • bookmar ks • a basic twitter clone • tagging • oembed suppor t • contact impor t • gr avatar suppor t (from vCard, Google or Yahoo) • photo management
  • 35.
    0.5 Structure • external apps • all on Google Code (SVN) • half written independently of Pinax • local apps • libraries (batteries included) • two sample projects • path hacking
  • 36.
  • 37.
  • 38.
  • 39.
  • 40.
  • 41.
    0.7 Release • translation removal • locations • frontend-admin and generic flat-blocks • account and group improvements • new task app • uni-form and Section 508 • better shared templates and media • more sample projects (and complete renamed social) • virtualenv and pip • pinax is now a package
  • 42.
    New Account App • turn OpenID on/off • turn sign up on/off • waiting list • invite only / private beta • token-based password reset • (resend signup email confirmation) • (use email for login)
  • 43.
    you signed upfor admin initiated each user has fixed user can invite beta account number to hand out anyone admin sent code user invited you general code unique invite just go to site (type in) (email link) have code external auth username / password username signup (no password) SIGNED UP
  • 44.
    New Group App • Don’t Repeat Yourself! • endo vs exo • group apps versus content apps • membership app
  • 45.
  • 46.
  • 47.
  • 48.
  • 49.
    Shared Templates andMedia • look up: • project • pinax new level (multiple themes) • app • media build command • ship with famfamfam silk icons
  • 50.
    Problems with 0.5structure • it largely relies on external dependencies being in svn and this is increasingly not the case (although it was when Pinax started) • it makes it difficult for Pinax itself to move away from svn • there is no management of dependencies between external dependencies, nor between particular projects in Pinax and their individual dependencies
  • 51.
    virtualenv and pip • encourage externals apps to be distutils- compliant • pip for package install (at least during development) • optional “batteries included” release • virtualenv to provide isolation, remove need for path hacking and just because it kicks ass
  • 52.
    release install (download tarball,extract and cd) $ python scripts/pinax-boot.py ../pinax-env $ source ../pinax-env/bin/activate (pinax-env)$ cd .. (pinax-env)$ pinax-admin clone_project social_project mysite (pinax-env)$ cd mysite/ (pinax-env)$ python manage.py syncdb (pinax-env)$ python manage.py runserver
  • 53.
    development install (download pinax-boot.pyfrom github) $ python pinax-boot.py --development ./pinax-env $ source ./pinax-env/bin/activate (pinax-env)$ (pinax-env)$ cd ./pinax-env/ (pinax-env)$ pip install --requirement src/pinax/requirements/external_apps.txt
  • 54.
  • 56.
    35 contributors 108 membersof pinax-core-dev list 58 at PyCon 336 users on code.pinaxproject.com 403 watchers on github 157 at PyCon 649 members of pinax-users list 398 at PyCon
  • 57.
    Pinax is NOTjust for social networks
  • 58.
    site site site site site site site site site site site site Learning Social Networks Intranets e-commerce Management Pinax Django
  • 59.
    site site site site site site site site site site site site Learning Social Networks Intranets e-commerce Management Pinax Django
  • 60.
    Editions • social networking • intranet • learning management • e-commerce • clubs and associations • conference management • software project management
  • 61.
    Roadmap • 0.7rc1out last week • 0.7 final in next day or two • 0.8 “translation release” based off 0.7 and in parallel to 0.9 • • 0.9 “big feature release” and invitations rethink friends: split into friends, contacts • • rethink profiles • workflow • open stack • calendar / events • new apps • new projects • new editions?
  • 62.
    Join Us DuringSprint (even if just in #pinax-sprint on freenode)
  • 63.