1. Motion
a application
Tonight’s presentation brought to you by
in association with The San Francisco Django Meetup Group

2. The Us
Ed Anuff Martin Atkins
Leah Culver Nima Badiey
Mike Malone Chris Hall
Brad Choate

3. A Live Demo

4. Local DB
Request Handler
typepad-motion
Django
Motion App
typepadapp
python-typepad-api
remoteobjects batchhttp
TypePad

5. Motion

6. Motion

7. It’s all BSD’d.

8. Feel free to utilize our teqniks
(a.k.a. steal our code)

9. We Feedback
But plz tell us if you
make it better!

10. Things we wanted
• Easy to install
• Easy to extend
• Easy to integrate

11. Easy to install
$ easy_install -q typepad-motion
$ django-admin.py typepadproject mymotion
> --settings=motion.settings
$ cat >> mymotion/local_settings.py
OAUTH_CONSUMER_KEY = 'key'
OAUTH_CONSUMER_SECRET = 'secret'
OAUTH_GENERAL_PURPOSE_KEY = 'gp_key'
OAUTH_GENERAL_PURPOSE_SECRET = 'gp_secret'
$ cd mymotion
$ ./manage.py syncdb
$ ./manage.py runserver

12. Easy to install
• PyPi & pip (or setuptools or distribute) for
package & dependency management
• Custom management commands
• typepadproject
• typepadapp
• Sane defaults

13. Management Commands
http://github.com/sixapart/typepadapp/blob/master/typepadapp/management/commands/typepadproject.py

14. Easy to extend
$ ./manage.py typepadapp myapp

15. Base Templates
$ cat motion/templates/motion/events.html
{% extends "motion/base/events.html" %}
$
http://github.com/sixapart/typepad-motion/tree/master/motion/templates/motion/

16. Template Caching
http://code.djangoproject.com/ticket/6262
http://github.com/sixapart/typepadapp/blob/master/typepadapp/cached_templates.py

17. Class-based Views
http://github.com/sixapart/typepadapp/blob/master/typepadapp/views/base.py#L61

18. Easy to integrate
It’s just a Django app. Not much more to say.

19. Replaced auth.models.User
http://github.com/sixapart/typepadapp/blob/master/typepadapp/models/users.py#L47
http://github.com/sixapart/typepadapp/blob/master/typepadapp/backends.py#L41

20. Below the

21. httplib2
• Developed by Joe Gregorio
• Sports a number of features that are useful
for talking to web services
• Caching
• Compression
• Extensible authentication mechanism

22. OAuth & httplib2
http://github.com/sixapart/python-typepad-api/blob/master/typepad/oauthclient.py#L106

23. OAuth & httplib2
>>> import oauthclient
>>> from oauth import oauth
>>> consumer = oauth.OAuthConsumer('key', 'secret')
>>> token = oauth.OAuthToken('key', 'secret')
>>> client = oauthclient.OAuthHttp()
>>> client.add_credentials(consumer, token, 'twitter.com')
>>> client.request(
'http://twitter.com/account/verify_credentials.json')
... THE RESPONSE ...

24. remoteobjects
• We work a lot with RESTish web services at Six Apart
• Mark Paschal (@markpasc) created an abstraction layer
that allows us to create robust client libraries very
easily
• The remoteobjects library allows you to describe an
API using a declarative syntax, like Django models
• Works very well with RESTful APIs, but it’s generic
enough to work with RPC APIs too

25. remoteobjects
class User(RemoteObject):
id = fields.Field()
name = fields.Field()
screen_name = fields.Field()
[ ... ]
followers_count = fields.Field()
status = fields.Object(‘status’)
@classmethod
def get_user(cls, http=None, **kwargs):
url = ‘/usrs/show’
if ‘id’ in kwargs:
url += ‘/%s.json’ % quote_plus(kwargs.pop(‘id’))
else:
url += ‘.json’
query = urlencode(kwargs)
url = urlunsplit((None, None, url, query, None))
return cls.get(urljoin(Twitter.endpoint, url),
http=http)

26. remoteobjects
>>> t = twitter.Twitter()
>>> t.add_credentials('mjmalone', 'thisisnotmypassword')
>>> friends_timeline = t.friends_timeline()
>>> [e.user.screen_name for e in friends.entries]
['ronaldheft', 'joestump', 'donttrythis',
'laughingsquid', 'FrankGruber', 'courtstarr', 'mbaratz',
'djchall', 'dozba', 'chrismessina', 'pop17', 'ijustine',
'calden', 'bryanveloso', 'jessenoller', 'pierre',
'optimarcusprime', 'snackfight', 'shiralazar']

27. remoteobjects
http://github.com/sixapart/remoteobjects

28. Batch HTTP

29. What?
• A mechanism for retrieving several distinct
resources in a single HTTP request.

30. TypePad API URLs
• The TypePad API is a REST API
• Each distinct resource has its own URL
which supports some subset of the
operations GET, PUT, POST and DELETE.

31. TypePad API URLs

32. TypePad API URLs
/users/apparentlymart
noun
id

33. TypePad API URLs
/users/apparentlymart/relationships
noun
id
sub-resource

34. TypePad API URLs
/users/apparentlymart/relationships/@by-group/6p1234123412341234
noun
id
sub-resource
parameterized
filter

35. TypePad API URLs
/users/apparentlymart/relationships/@following
noun
id
sub-resource
boolean
filter

36. Motion’s Home Page
All of the data here
comes from the
TypePad API

37. Motion’s Home Page
/users/apparentlymart
/users/apparentlymart/relationships/@following
/users/apparentlymart/relationships/@follower
Requires 5 resources
from api.typepad.com
/groups/{id}/events /groups/{id}/memberships

38. How can we retrieve
those five resource
from TypePad as quickly
and efficiently as
possible?

39. Design Goals
• Retrieve all required resources in parallel
• Allow TypePad to optimize the multi-get to
return a response as quickly as possible
• Be consistent with how single resources
are returned in the normal case
• Invent as little as possible

40. Parallel HTTP Requests
• Reasonably easy from the client’s
perspective, assuming that their HTTP
client library doesn’t suck.
• Difficult to optimize predictably on
TypePad’s end because we can’t control the
order and timing of the handling of
individual requests.

41. HTTP Pipelining
• Poor support from libraries and
infrastructure.
• Responses must be returned in order of
request, which complicates the API
dispatcher.

42. Batch Processing API

43. Batch Processing API
Client
Batch Processor
API Endpoint API Endpoint

44. Batch Processing API
• GData (ab?)uses Atom to define a list of
operations.
• AWS uses some AWS-specific query string
arguments or SOAP-ish stuff.
• Ideally we wanted batch processing to be
an orthogonal layer.

45. Batch Processing API
• Long story short: we designed a batch
processing mechanism that can hopefully
apply to any REST-based API.

46. Batch Processing API
• Batch Processor is conceptually similar to
an HTTP proxy.
• The frontend accepts a POST request
containing a MIME multipart message
containing a bunch of requests, and returns
a multipart message containing the
corresponding response.

47. Batch Processing API
• Conceptually, the backend of the batch
processor makes its own HTTP requests to
the indicated resources like an HTTP proxy
would.
• In practice, TypePad’s implementation
handles the entire batch job together and
fakes up the separate responses.

48. Batch Processing API
• We’ve published a spec and some open
source sample implementations in the hope
that it’s useful to the community:
http://martin.atkins.me.uk/specs/batchhttp
http://github.com/sixapart/batchhttp
http://github.com/sixapart/libhttp-request-multi-perl

49. Python Batch Request
def callback(url, subresponse, subcontent):
self.subresponse = subresponse
self.subcontent = subcontent
client = BatchClient(endpoint=
'http://127.0.0.1:8000/batch-processor')
client.batch_request()
client.batch({'uri': 'http://example.com/moose'},
callback=callback)
client.batch({'uri': 'http://example.com/mouse'},
callback=callback)
client.complete_batch()

50. That’s All!
http://developer.typepad.com/
http://github.com/sixapart/typepad-motion/
Questions?