Does your stuff scale?
Steven Holmes
http://p.ota.to
Potato?
Does your stuff scale?
1. Django + Google App Engine
2. Scaling an organisation + culture
Non-technical
Scalability
Technical
1. Load
2. Functional
3. Organisational
4. Geographic
Part 1
+
1. Auto-scaling
2. Services & APIs
3. No sysadmin
Why Google App Engine?
Just make good things.
App Engine: caveats
1. Sandbox
2. File system
3. Portability
What is it?
• Django ported to NoSQL
• GitHub, open source
• Django 1.4, Django 1.5 beta
Pros
• Familiar APIs
• Portability
• Works in production
How do you run it?
• Clone repo(s)
• ./manage.py runserver
Cons
• Familiarity can be misleading
• Can feel ‘heavy’ / kludgy
Django non-rel github.com/django-nonrel
What is it?
• Lightweight skeleton
• Django + App Engine models
Pros
• “Best of both worlds”
• NDB
• Automatic caching, structured data
How do you run it?
• Clone repo
• dev_appserver.py .
Cons
• Learning curve
• Portability
Djappengine github.com/potatolondon/djappengine
What is it?
• Custom Django database backend wrapper for Django's MySQL backend
Pros
• Fully supported Django
• Flexibility
• Unpredictable reporting
How do you run it?
• Create a CloudSQL instance
• 'google.appengine.ext.django.backends.rdbms' in settings.py
• ./manage.py syncdb
Cons
• More setup
• Possibly not as scalable as datastore?
App Engine + CloudSQL Search “Django + CloudSQL”
Part 2
Technical
Non-technical
Scalability
1. Load
2. Functional
3. Organisational
4. Geographic
+
• Planning
• Caching
• Offline tasks
• Preparation
◦ Load tests
◦ Profiling (e.g. using app stats)
Scalability - Load
Technical
Non-technical
Scalability
1. Load
2. Functional
3. Organisational
4. Geographic
+
• Django!
◦ Documentation, Community
• App Engine
◦ Services & APIs
■ Memcache
■ Taskqueue, mapreduce
■ Search
■ Email
■ Images
◦ Versioning
■ 10 testable versions per app
■ http://0.myapp.appspot.com {live}
■ http://1.myapp.appspot.com {not live}
■ http://some-feature-branch.myapp.appspot.com {not live}
■ A/B testing / traffic splitting
◦ SDK
Scalability - Functional
Technical
Non-technical
Scalability
1. Load
2. Functional
3. Organisational
4. Geographic
+
• Be a minimalist
• Remove bottlenecks & overhead
• Just make good things.
• Internal apps
Scalability - Organisational
• Be a minimalist
• Remove bottlenecks & overhead
• Just make good things.
• Internal apps
◦ e.g. wiki, recruitment, deployments, checklists
Scalability - Organisational
• Integrated and centralised
◦ Google Apps domain
• Security
◦ HTTPs
Technical
Non-technical
Scalability
1. Load
2. Functional
3. Organisational
4. Geographic
+
Just make good things.
Questions?
• Potato lab > http://p.ota.to/lab
• GitHub > github.com/potatolondon
• App Engine > developers.google.com/appengine
• Django non-rel > django-nonrel.org

Does Your Stuff Scale?

  • 1.
    Does your stuffscale? Steven Holmes http://p.ota.to
  • 2.
  • 3.
    Does your stuffscale? 1. Django + Google App Engine 2. Scaling an organisation + culture
  • 4.
  • 5.
  • 6.
    1. Auto-scaling 2. Services& APIs 3. No sysadmin Why Google App Engine?
  • 7.
  • 8.
    App Engine: caveats 1.Sandbox 2. File system 3. Portability
  • 9.
    What is it? •Django ported to NoSQL • GitHub, open source • Django 1.4, Django 1.5 beta Pros • Familiar APIs • Portability • Works in production How do you run it? • Clone repo(s) • ./manage.py runserver Cons • Familiarity can be misleading • Can feel ‘heavy’ / kludgy Django non-rel github.com/django-nonrel
  • 11.
    What is it? •Lightweight skeleton • Django + App Engine models Pros • “Best of both worlds” • NDB • Automatic caching, structured data How do you run it? • Clone repo • dev_appserver.py . Cons • Learning curve • Portability Djappengine github.com/potatolondon/djappengine
  • 13.
    What is it? •Custom Django database backend wrapper for Django's MySQL backend Pros • Fully supported Django • Flexibility • Unpredictable reporting How do you run it? • Create a CloudSQL instance • 'google.appengine.ext.django.backends.rdbms' in settings.py • ./manage.py syncdb Cons • More setup • Possibly not as scalable as datastore? App Engine + CloudSQL Search “Django + CloudSQL”
  • 15.
  • 16.
  • 17.
    • Planning • Caching •Offline tasks • Preparation ◦ Load tests ◦ Profiling (e.g. using app stats) Scalability - Load
  • 18.
  • 19.
    • Django! ◦ Documentation,Community • App Engine ◦ Services & APIs ■ Memcache ■ Taskqueue, mapreduce ■ Search ■ Email ■ Images ◦ Versioning ■ 10 testable versions per app ■ http://0.myapp.appspot.com {live} ■ http://1.myapp.appspot.com {not live} ■ http://some-feature-branch.myapp.appspot.com {not live} ■ A/B testing / traffic splitting ◦ SDK Scalability - Functional
  • 20.
  • 21.
    • Be aminimalist • Remove bottlenecks & overhead • Just make good things. • Internal apps Scalability - Organisational
  • 25.
    • Be aminimalist • Remove bottlenecks & overhead • Just make good things. • Internal apps ◦ e.g. wiki, recruitment, deployments, checklists Scalability - Organisational • Integrated and centralised ◦ Google Apps domain • Security ◦ HTTPs
  • 26.
  • 27.
  • 28.
    Questions? • Potato lab> http://p.ota.to/lab • GitHub > github.com/potatolondon • App Engine > developers.google.com/appengine • Django non-rel > django-nonrel.org