Pylons + Tokyo Cabinet
by Ben Cheng from Oursky.com

About Us
• Founder of Oursky Ltd.
• Web application development services
(Ruby on Rails, Django, Pylons, PHP,
Wordpress, etc.);
• FONCEPT.com - a T-Shirt design sharing
portal;
• Working on a SaaS workflow application.

Pylons

What is Pylons?
• Flexible, Modular, Extensible
• Built on established libraries
• Close to 1.0
• Foundation of TurboGears 2.0

Who is using Pylons?
• Bittorrent
• Reddit (code.reddit.com!)
• and hundreds of sites

WSGI
• WSGI (Web Server Gateway Interface)
• Wrap a “app” inside another “app”
• Middleware, e.g.:
• Authentication
• Beaker (Cache, Session)
• Routes

The Stack
• Models: SQLAlchemy, CouchDB...
• Templates: Mako, Genshi, Jinja2, ...
• Helpers: Web Helpers, FormAlchemy, ...
• Request Dispatching: Routes, ...
• Session / Caching: Beaker, ...
• Deployment: Paste, mod_wsgi, Twisted, ...

Take advantage of
Python Power
• TIOBE: Python is no. 8
• Such as:
• Want interactive debugger? iPython
• Logging? Chainsaw
• Testing? Nose, MiniMock
• ..... blah blah balh

Power of other Python
Web Framework
• Tons of lib from Zope
• ToscaWidgets from TurboGears
• Authkit
• (except Django)

Community
• Pylons Book
• Quick response (e.g.):
• We fired Ticket #620 at Pylons Trac
• “init_model called twice in nosetests”
• Bug get fixed in 2 weeks, committed to
trunk and temp. solution are discussed.

Tokyo Cabinet

Introduction
• By mixi.jp (Facebook in Japan)
• TokyoCabinet (Engine)
• TokyoTyrant (Server)
• TokyoDystopia (Full-Text Search)
• Opensources, standard C

TokyoCabinet
• Key/value DB
• Insert: 0.4 sec / 1M records
• Search: 0.33 sec / 1M records
• hash / B+ Tree structure
• Key Value / Hash / B+ Tree / Fixed Length /
Table
• Driver: Perl, Ruby, Python, PHP, Erlang...

TokyoTyrant
• Network server
• Binary / Memcache / HTTP protocol
• 17.2 sec / 1M queries
• Hot Backup, Master-to-Master replication
• Extensible by Lua

Lua - Extensions
function incr(key, i)
i = tonumber(i)
local old = tonumber(_get(key))
i = old + i
_put(key, i)
return i
end

Lua - Extension
function expire()
local args={}
local cdata = string.format(“%d”, _time())
table.insert(args, “addcond0x0NUMLE”..cdata)
table.insert(args, “out”)
local res = _misc(“search”, args)
if not res then
_log(“expire failed”)
end
end

Extensible to...
• Inverted index
• Set operations
• (and we use it to do revision management)

LightCloud
• Distributed and persistent key-value
database
• Opensource and used by Plurk
• Scale out by just adding nodes
• Nodes are replicated
Elapsed for 10000 gets: 1.74538516998 seconds [memcache]
Elapsed for 10000 gets: 3.57339096069 seconds [lightcloud]
Elapsed for 10000 sets: 1.88236999512 seconds [memcache]
Elapsed for 10000 sets: 9.23674893379 seconds [lightcloud]

Links
• http://www.pylonshq.com
• http://opensource.plurk.com/LightCloud/
• http://1978th.net/tokyocabinet/
• Reach me at bencheng@oursky.com