在QCon Beijing 2010上的演讲

1. Python
Web 2.0
QCon Beijing 2010
http://www.flickr.com/photos/arnolouise/2986467632/

2. About Me
• Python
• 2002 Python
• 2004
Python
• http://www.douban.com/
people/hongqn/
• hongqn@douban.com
• http://twitter.com/hongqn

3. Python
• Python is a programming language that lets you
work more quickly and integrate your systems more
effectively. You can learn to use Python and see
almost immediate gains in productivity and lower
maintenance costs. (via http://python.org/)

4. Languages in
C
27%
Javascript
12%
C++
Python 3%
58%
(Pyrex/R/Erlang/Go/Shell)
1%

5. Why Python?

7. • Hello World: 1

8. • Hello World: 1
• :1

9. • Hello World: 1
• :1
• :1

10. • Hello World: 1
• :1
• :1
• :3

12. : 13

13. : 13
import os
from collections import defaultdict
d = defaultdict(int)
for dirpath, dirnames, filenames in os.walk('.'):
for filename in filenames:
path = os.path.join(dirpath, filename)
ext = os.path.splitext(filename)[1]
d[ext] += len(list(open(path)))
for ext, n_lines in d.items():
print ext, n_lines

14. •
• Pythonic

15. •
1. svn ci
2. svn up
3. restart

16. • Web
•
•
•

17. • Battery Included: 200+
• PyPI: 9613 packages currently
• / / / / / /
/...
• easily extensible

18. Python

19. Python

20. Python
Just kidding :-p

22. Web Server

23. Web Server
• python -m SimpleHTTPServer

24. Web Server
• python -m SimpleHTTPServer

25. web.py http://webpy.org/
import web
urls = (
'/(.*)', 'hello'
)
app = web.application(urls, globals())
class hello:
def GET(self, name):
if not name:
name = 'World'
return 'Hello, ' + name + '!'
if __name__ == "__main__":
app.run()

26. Flask http://flask.pocoo.org/
import flask import Flask
app = Flask(__name__)
@app.route("/<name>")
def hello(name):
if not name:
name = 'World'
return 'Hello, ' + name + '!'
if __name__ == "__main__":
app.run()

27. WSGI
http://www.python.org/dev/peps/pep-0333/

28. Why so many Python
web frameworks?
• Because you can write your own
framework in 3 hours and a total of 60
lines of Python code.
• http://bitworking.org/news/
Why_so_many_Python_web_frameworks

29. doctest
def cube(x):
"""
>>> cube(10)
1000
"""
return x * x
def _test():
import doctest
doctest.testmod()
if __name__ == "__main__":
_test()

30. nose http://somethingaboutorange.com/mrl/projects/nose/
from cube import cube
def test_cube():
result = cube(10)
assert result == 1000

31. numpy http://numpy.scipy.org/
>>> from numpy import *
>>> A = arange(4).reshape(2, 2)
>>> A
array([[0, 1],
[2, 3]])
>>> dot(A, A.T)
array([[ 1, 3],
[ 3, 13]])

32. ipython http://numpy.scipy.org/
$ ipython -pylab
In [1]: X = frange(0, 10, 0.1)
In [2]: Y = [sin(x) for x in X]
In [3]: plot(X, Y)

33. ipython http://numpy.scipy.org/
$ ipython -pylab
In [1]: X = frange(0, 10, 0.1)
In [2]: Y = [sin(x) for x in X]
In [3]: plot(X, Y)

34. virtualenv http://virtualenv.openplans.org/
python
$ python go-pylons.py --no-site-packages mydevenv
$ cd mydevenv
$ source bin/activate
(mydevenv)$ paster create -t new9 helloworld

35. Pyrex/Cython
cdef extern from "math.h"
double sin(double)
cdef double f(double x):
return sin(x*x)

36. Pythonic

37. >>> import this
The Zen of Python, by Tim Peters http://bit.ly/pyzencn
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to
break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.

38. In the face of ambiguity, refuse the
temptation to guess.
There should be one-- and
preferably only one --obvious way
to do it.
Although that way may not be Python
obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than
*right* now.
If the implementation is hard to
explain, it's a bad idea.
If the implementation is easy to
explain, it may be a good idea.
Namespaces are one honking great
idea -- let's do more of those!

39. Simple is better than
complex
class HelloWorld
{
public static void main(String args[])
{
System.out.println("Hello World!");
}
}

40. Simple is better than
complex
print "Hello World!"

41. Readability counts

42. Readability counts
• {} end

43. Readability counts
• {} end
• (except "@" for
decorators)

44. Readability counts
• {} end
• (except "@" for
decorators)
if limit is not None and len(ids)>limit:
ids = random.sample(ids, limit)

45. TOOWTDI
• There (should be) Only One Way To Do It.
• vs. Perlish TIMTOWTDI (There Is More
Than One Way To Do It)

46. TOOWTDI
• There (should be) Only One Way To Do It.
• vs. Perlish TIMTOWTDI (There Is More
Than One Way To Do It)
a = [1, 2, 3, 4, 5]
b = []
for i in range(len(a)):
b.append(a[i]*2)

47. TOOWTDI
• There (should be) Only One Way To Do It.
• vs. Perlish TIMTOWTDI (There Is More
Than One Way To Do It)
a = [1, 2, 3, 4, 5]
b = []
for i in range(len(a)):
b.append(a[i]*2)

48. TOOWTDI
• There (should be) Only One Way To Do It.
• vs. Perlish TIMTOWTDI (There Is More
Than One Way To Do It)
a = [1, 2, 3, 4, 5] b = []
b = [] for x in a:
for i in range(len(a)):
b.append(x*2)
b.append(a[i]*2)

49. TOOWTDI
• There (should be) Only One Way To Do It.
• vs. Perlish TIMTOWTDI (There Is More
Than One Way To Do It)
a = [1, 2, 3, 4, 5] b = []
b = [] for x in a:
for i in range(len(a)):
b.append(x*2)
b.append(a[i]*2)

50. TOOWTDI
• There (should be) Only One Way To Do It.
• vs. Perlish TIMTOWTDI (There Is More
Than One Way To Do It)
b = [x*2 for x in a]

51. http://twitter.com/robbinfan/status/9879724095
http://twitter.com/hongqn/status/9883515681

52. Python C
http://www.flickr.com/photos/nicksieger/281055485/ http://www.flickr.com/photos/nicksieger/281055530/

53. Ruby
http://www.flickr.com/photos/nicksieger/280661836/

54. Java
http://www.flickr.com/photos/nicksieger/280662707/

55. Python

57. • svn

58. • svn
• ( list)

59. • svn
• ( list)
• +

60. • svn
• ( list)
• +
• parser

61. config.py
MEMCACHED_ADDR = ['localhost:11211']
from local_config import *

62. config.py
MEMCACHED_ADDR = ['localhost:11211']
from local_config import *
local_config.py
MEMCACHED_ADDR = [
'frodo:11211',
'sam:11211',
'pippin:11211',
'merry:11211',
]

63. config.py
MEMCACHED_ADDR = ['localhost:11211']
from local_config import *
.py local_config.py
exec MEMCACHED_ADDR = [
'frodo:11211',
'sam:11211',
'pippin:11211',
'merry:11211',
]

64. •

65. class GroupUI(object):
def new_topic(self, request):
if self.group.can_post(request.user):
return new_topic_ui(self.group)
else:
request.response.set_status(403, "Forbidden")
return error_403_ui(msg=" ")
def join(self, request):
if self.group.can_join(request.user):
...
class Group(object):
def can_post(self, user):
return self.group.has_member(user)
def can_join(self, user):
return not self.group.has_banned(user)

66. class GroupUI(object):
@check_permission('post', msg=" ")
def new_topic(self, request):
return new_topic_ui(self.group)
@check_permission('join', msg=" ")
def join(self, request):
...
class Group(object):
def can_post(self, user):
return self.group.has_member(user)
def can_join(self, user):
return not self.group.has_banned(user)

67. decorator
def print_before_exec(func):
def _(*args, **kwargs):
print "decorated"
return func(*args, **kwargs)
return _
@print_before_exec
def double(x):
print x*2
double(10)

68. decorator
def print_before_exec(func):
def _(*args, **kwargs):
print "decorated"
return func(*args, **kwargs)
return _
decorated
@print_before_exec
def double(x): 20
print x*2
double(10)

69. class check_permission(object):
def __init__(self, action, msg=None):
self.action = action
self.msg = msg
def __call__(self, func):
def _(ui, req, *args, **kwargs):
f = getattr(ui.perm_obj, 'can_' + self.action)
if f(req.user):
return func(ui, *args, **kwargs)
raise BadPermission(ui.perm_obj, self.action,
self.msg)
return _

70. class check_permission(object):
def __init__(self, action, msg=None):
self.action = action
self.msg = msg
def __call__(self, func):
def _(ui, req, *args, **kwargs):
f = getattr(ui.perm_obj, 'can_' + self.action)
if f(req.user):
return func(ui, *args, **kwargs)
raise BadPermission(ui.perm_obj, self.action,
self.msg)
return _

71. class check_permission(object):
def __init__(self, action, msg=None):
self.action = action
self.msg = msg
def __call__(self, func):
def _(ui, req, *args, **kwargs):
f = getattr(ui.perm_obj, 'can_' + self.action)
if f(req.user):
return func(ui, *args, **kwargs)
raise BadPermission(ui.perm_obj, self.action,
self.msg)
return _

72. class check_permission(object):
def __init__(self, action, msg=None):
self.action = action
self.msg = msg
def __call__(self, func):
def _(ui, req, *args, **kwargs):
f = getattr(ui.perm_obj, 'can_' + self.action)
if f(req.user):
return func(ui, *args, **kwargs)
raise BadPermission(ui.perm_obj, self.action,
self.msg)
return _

73. class check_permission(object):
def __init__(self, action, msg=None):
self.action = action
self.msg = msg
def __call__(self, func):
def _(ui, req, *args, **kwargs):
f = getattr(ui.perm_obj, 'can_' + self.action)
if f(req.user):
return func(ui, *args, **kwargs)
raise BadPermission(ui.perm_obj, self.action,
self.msg)
return _

74. class check_permission(object):
def __init__(self, action, msg=None):
self.action = action
self.msg = msg
def __call__(self, func):
def _(ui, req, *args, **kwargs):
f = getattr(ui.perm_obj, 'can_' + self.action)
if f(req.user):
return func(ui, *args, **kwargs)
raise BadPermission(ui.perm_obj, self.action,
self.msg)
return _

75. class GroupUI(object):
@check_permission('post', msg=" ")
def new_topic(self, request):
return new_topic_ui(self.group)
@check_permission('join', msg=" ")
def join(self, request):
...
class Group(object):
def can_post(self, user):
return self.group.has_member(user)
def can_join(self, user):
return not self.group.has_banned(user)

76. •

77. def send_notification_mail(email, subject, body):
msg = MSG_SEND_MAIL + '0' + email + '0' + subject + '0' + body
mq.put(msg)
def async_worker():
msg = mq.get()
msg = msg.split('0')
cmd = msg[0]
if cmd == MSG_SEND_MAIL:
email, subject, body = msg[1:]
fromaddr = 'no-reply@douban.com'
email_body = make_email_body(fromaddr, email, subject, body)
smtp = smtplib.SMTP('mail')
smtp.sendmail(fromaddr, email, email_body)
elif cmd == MSG_xxxx:
...
elif cmd == MSG_yyyy:
...

78. @async
def send_notification_mail(email, subject, body):
fromaddr = 'no-reply@douban.com'
email_body = make_email_body(fromaddr, email, subject,
body)
smtp = smtplib.SMTP('mail')
smtp.sendmail(fromaddr, email, email_body)

79. def async(func):
mod = sys.modules[func.__module__]
fname = 'origin_' + func.__name__
mod.__dict__[fname] = func
def _(*a, **kw):
body = cPickle.dumps((mod.__name__, fname, a, kw))
mq.put(body)
return _
def async_worker():
modname, fname, a, kw = cPickle.loads(mq.get())
__import__(modname)
mod = sys.modules[modname]
mod.__dict__[fname](*a, **kw)

80. def async(func):
mod = sys.modules[func.__module__]
fname = 'origin_' + func.__name__
mod.__dict__[fname] = func
def _(*a, **kw):
body = cPickle.dumps((mod.__name__, fname, a, kw))
mq.put(body)
return _
def async_worker():
modname, fname, a, kw = cPickle.loads(mq.get())
__import__(modname)
mod = sys.modules[modname]
mod.__dict__[fname](*a, **kw)

81. def async(func):
mod = sys.modules[func.__module__]
fname = 'origin_' + func.__name__
mod.__dict__[fname] = func
def _(*a, **kw):
body = cPickle.dumps((mod.__name__, fname, a, kw))
mq.put(body)
return _
def async_worker():
modname, fname, a, kw = cPickle.loads(mq.get())
__import__(modname)
mod = sys.modules[modname]
mod.__dict__[fname](*a, **kw)

82. def async(func):
mod = sys.modules[func.__module__]
fname = 'origin_' + func.__name__
mod.__dict__[fname] = func
def _(*a, **kw):
body = cPickle.dumps((mod.__name__, fname, a, kw))
mq.put(body)
return _
def async_worker():
modname, fname, a, kw = cPickle.loads(mq.get())
__import__(modname)
mod = sys.modules[modname]
mod.__dict__[fname](*a, **kw)

83. def async(func):
mod = sys.modules[func.__module__]
fname = 'origin_' + func.__name__
mod.__dict__[fname] = func
def _(*a, **kw):
body = cPickle.dumps((mod.__name__, fname, a, kw))
mq.put(body)
return _
def async_worker():
modname, fname, a, kw = cPickle.loads(mq.get())
__import__(modname)
mod = sys.modules[modname]
mod.__dict__[fname](*a, **kw)

84. def async(func):
mod = sys.modules[func.__module__]
fname = 'origin_' + func.__name__
mod.__dict__[fname] = func
def _(*a, **kw):
body = cPickle.dumps((mod.__name__, fname, a, kw))
mq.put(body)
return _
def async_worker():
modname, fname, a, kw = cPickle.loads(mq.get())
__import__(modname)
mod = sys.modules[modname]
mod.__dict__[fname](*a, **kw)

85. def async(func):
mod = sys.modules[func.__module__]
fname = 'origin_' + func.__name__
mod.__dict__[fname] = func
def _(*a, **kw):
body = cPickle.dumps((mod.__name__, fname, a, kw))
mq.put(body)
return _
def async_worker():
modname, fname, a, kw = cPickle.loads(mq.get())
__import__(modname)
mod = sys.modules[modname]
mod.__dict__[fname](*a, **kw)

86. def async(func):
mod = sys.modules[func.__module__]
fname = 'origin_' + func.__name__
mod.__dict__[fname] = func
def _(*a, **kw):
body = cPickle.dumps((mod.__name__, fname, a, kw))
mq.put(body)
return _
def async_worker():
modname, fname, a, kw = cPickle.loads(mq.get())
__import__(modname)
mod = sys.modules[modname]
mod.__dict__[fname](*a, **kw)

87. • cache (SQL, , etc)

88. def get_latest_review_id():
review_id = mc.get('latest_review_id')
if review_id is None:
review_id = exc_sql("select max(id) from review")
mc.set('latest_review_id', review_id)
return review_id

89. @cache('latest_review_id')
def get_latest_review_id():
return exc_sql("select max(id) from review")

90. def cache(key):
def deco(func):
def _(*args, **kwargs):
r = mc.get(key)
if r is None:
r = func(*args, **kwargs)
mc.set(key, r)
return r
return _
return deco

91. def cache(key):
def deco(func):
def _(*args, **kwargs):
r = mc.get(key)
if r is None:
r = func(*args, **kwargs)
mc.set(key, r)
return r
return _
return deco

92. cache key
def get_review(id):
key = 'review:%s' % id
review = mc.get(key)
if review is None: # cache miss
id, author_id, text = exc_sql("select id, author_id,
text from review where id=%s", id)
review = Review(id, author_id, text)
mc.set(key, review)
return review

93. cache key
decorator
@cache('review:{id}')
def get_review(id):
id, author_id, text = exc_sql("select id, author_id,
text from review where id=%s", id)
return Review(id, author_id, text)

94. def cache(key_pattern, expire=0):
def deco(f):
arg_names, varargs, varkw, defaults = inspect.getargspec(f)
if varargs or varkw:
raise Exception("not support varargs")
gen_key = gen_key_factory(key_pattern, arg_names, defaults)
def _(*a, **kw):
key = gen_key(*a, **kw)
r = mc.get(key)
if r is None:
r = f(*a, **kw)
mc.set(key, r, expire)
return r
return _
return deco

95. def cache(key_pattern, expire=0):
def deco(f):
arg_names, varargs, varkw, defaults = inspect.getargspec(f)
if varargs or varkw:
raise Exception("not support varargs")
gen_key = gen_key_factory(key_pattern, arg_names, defaults)
def _(*a, **kw):
key = gen_key(*a, **kw)
r = mc.get(key)
if r is None:
r = f(*a, **kw)
mc.set(key, r, expire)
return r
return _
return deco

96. def cache(key_pattern, expire=0):
def deco(f):
arg_names, varargs, varkw, defaults = inspect.getargspec(f)
if varargs or varkw:
raise Exception("not support varargs")
gen_key = gen_key_factory(key_pattern, arg_names, defaults)
def _(*a, **kw):
key = gen_key(*a, **kw)
r = mc.get(key)
if r is None:
r = f(*a, **kw)
mc.set(key, r, expire)
return r
return _
return deco

97. inspect.getargspec
>>> import inspect
>>> def f(a, b=1, c=2):
... pass
...
>>> inspect.getargspec(f)
ArgSpec(args=['a', 'b', 'c'], varargs=None, keywords=None,
defaults=(1, 2))
>>>
>>>
>>> def f(a, b=1, c=2, *args, **kwargs):
... pass
...
>>> inspect.getargspec(f)
ArgSpec(args=['a', 'b', 'c'], varargs='args', keywords='kwargs',
defaults=(1, 2))

98. def cache(key_pattern, expire=0):
def deco(f):
arg_names, varargs, varkw, defaults = inspect.getargspec(f)
if varargs or varkw:
raise Exception("not support varargs")
gen_key = gen_key_factory(key_pattern, arg_names, defaults)
def _(*a, **kw):
key = gen_key(*a, **kw)
r = mc.get(key)
if r is None:
r = f(*a, **kw)
mc.set(key, r, expire)
return r
return _
return deco

99. def cache(key_pattern, expire=0):
def deco(f):
arg_names, varargs, varkw, defaults = inspect.getargspec(f)
if varargs or varkw:
raise Exception("not support varargs")
gen_key = gen_key_factory(key_pattern, arg_names, defaults)
def _(*a, **kw):
key = gen_key(*a, **kw)
r = mc.get(key)
if r is None:
hint:
r = f(*a, **kw)
mc.set(key, r, expire)
return r • str.format in python 2.6:
'{id}'.format(id=1) => '1'
return _
return deco • dict(zip(['a', 'b', 'c'], [1, 2, 3]))
=> {'a': 1, 'b': 2, 'c': 3}

100. def cache(key_pattern, expire=0):
def deco(f):
arg_names, varargs, varkw, defaults = inspect.getargspec(f)
if varargs or varkw:
raise Exception("not support varargs")
gen_key = gen_key_factory(key_pattern, arg_names, defaults)
def _(*a, **kw):
key = gen_key(*a, **kw)
r = mc.get(key)
if r is None:
hint:
r = f(*a, **kw)
mc.set(key, r, expire)
return r • str.format in python 2.6:
'{id}'.format(id=1) => '1'
return _
return deco • dict(zip(['a', 'b', 'c'], [1, 2, 3]))
=> {'a': 1, 'b': 2, 'c': 3}

101. def cache(key_pattern, expire=0):
def deco(f):
arg_names, varargs, varkw, defaults = inspect.getargspec(f)
if varargs or varkw:
raise Exception("not support varargs")
gen_key = gen_key_factory(key_pattern, arg_names, defaults)
def _(*a, **kw):
key = gen_key(*a, **kw)
r = mc.get(key)
if r is None:
r = f(*a, **kw)
mc.set(key, r, expire)
return r
return _
return deco

102. • feed feed
entry_id

103. class Feed(object):
def get_entries(self, limit=10):
ids = exc_sqls("select id from entry where feed_id=
%s order by id desc limit %s", (self.id, limit))
return [Entry.get(id) for id in ids]
class FeedCollection(object):
def get_entries(self, limit=10):
mixed_entries = []
for feed in self.feeds:
entries = feed.get_entries(limit=limit)
mixed_entries += entries
mixed_entries.sort(key=lambda e: e.id, reverse=True)
return mixed_entries[:10]

104. class Feed(object):
def get_entries(self, limit=10):
ids = exc_sqls("select id from entry where feed_id=
%s order by id desc limit %s", (self.id, limit))
return [Entry.get(id) for id in ids]
class FeedCollection(object):
def get_entries(self, limit=10):
mixed_entries = []
for feed in self.feeds:
entries = feed.get_entries(limit=limit)
mixed_entries += entries
mixed_entries.sort(key=lambda e: e.id, reverse=True)
return mixed_entries[:10]

105. class Feed(object):
def get_entries(self, limit=10):
ids = exc_sqls("select id from entry where feed_id=
%s order by id desc limit %s", (self.id, limit))
return [Entry.get(id) for id in ids]
class FeedCollection(object):
def get_entries(self, limit=10):
mixed_entries = []
for feed in self.feeds:
entries = feed.get_entries(limit=limit)
mixed_entries += entries
mixed_entries.sort(key=lambda e: e.id, reverse=True)
return mixed_entries[:10]

106. class Feed(object):
def get_entries(self, limit=10):
ids = exc_sqls("select id from entry where feed_id=
%s order by id desc limit %s", (self.id, limit))
return [Entry.get(id) for id in ids]
class FeedCollection(object):
=
def get_entries(self, limit=10):
len(self.feeds) * limit
mixed_entries = []
for feed in self.feeds:
entries = feed.get_entries(limit=limit)
mixed_entries += entries
mixed_entries.sort(key=lambda e: e.id, reverse=True)
return mixed_entries[:10]

107. class Feed(object):
def get_entries(self, limit=10):
ids = exc_sqls("select id from entry where feed_id=
%s order by id desc limit %s", (self.id, limit))
return [Entry.get(id) for id in ids]
class FeedCollection(object):
Entry.get =
def get_entries(self, limit=10):
len(self.feeds-1) * limit
mixed_entries = []
for feed in self.feeds:
entries = feed.get_entries(limit=limit)
mixed_entries += entries
mixed_entries.sort(key=lambda e: e.id, reverse=True)
return mixed_entries[:10]

108. iterator and generator
def fib():
x, y = 1, 1
while True:
yield x
x, y = y, x+y
def odd(seq):
return (n for n in seq if n%2)
def less_than(seq, upper_limit):
for number in seq:
if number >= upper_limit:
break
yield number
print sum(odd(less_than(fib(), 4000000)))

109. itertools
• count([n]) --> n, n+1, n+2
• cycle(p) --> p0, p1, ... plast, p0, p1, ...
• repeat(elem [,n]) --> elem, elem, elem, ...
endless or up to n times
• izip(p, q, ...) --> (p[0], q[0]), (p[1],
q[1]), ...
• islice(seq, [start,] stop [, step]) -->
elements from seq[start:stop:step]
• ... and more ...

110. class Feed(object):
def iter_entries(self):
start_id = sys.maxint
while True:
entry_ids = exc_sqls("select id from entry where
feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id))
if not entry_ids:
break
for entry_id in entry_ids:
yield Entry.get(entry_id)
start_id = entry_ids[-1]
class FeedCollection(object):
def iter_entries(self):
return imerge(*[feed.iter_entries() for feed in self.feeds])
def get_entries(self, limit=10):
return list(islice(self.iter_entries(), limit))

111. class Feed(object):
def iter_entries(self):
start_id = sys.maxint
while True:
entry_ids = exc_sqls("select id from entry where
feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id))
if not entry_ids:
break
for entry_id in entry_ids:
yield Entry.get(entry_id)
start_id = entry_ids[-1]
class FeedCollection(object):
def iter_entries(self):
return imerge(*[feed.iter_entries() for feed in self.feeds])
def get_entries(self, limit=10):
return list(islice(self.iter_entries(), limit))

112. class Feed(object):
def iter_entries(self):
start_id = sys.maxint
while True:
entry_ids = exc_sqls("select id from entry where
feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id))
if not entry_ids:
break
for entry_id in entry_ids:
yield Entry.get(entry_id)
start_id = entry_ids[-1]
class FeedCollection(object):
def iter_entries(self):
return imerge(*[feed.iter_entries() for feed in self.feeds])
def get_entries(self, limit=10):
return list(islice(self.iter_entries(), limit))

113. class Feed(object):
def iter_entries(self):
start_id = sys.maxint
while True:
entry_ids = exc_sqls("select id from entry where
feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id))
if not entry_ids:
break
for entry_id in entry_ids:
yield Entry.get(entry_id)
start_id = entry_ids[-1]
class FeedCollection(object):
def iter_entries(self):
return imerge(*[feed.iter_entries() for feed in self.feeds])
def get_entries(self, limit=10):
return list(islice(self.iter_entries(), limit))

114. class Feed(object):
def iter_entries(self):
start_id = sys.maxint
while True:
entry_ids = exc_sqls("select id from entry where
feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id))
if not entry_ids:
break
for entry_id in entry_ids:
yield Entry.get(entry_id)
start_id = entry_ids[-1]
class FeedCollection(object):
def iter_entries(self):
return imerge(*[feed.iter_entries() for feed in self.feeds])
def get_entries(self, limit=10):
return list(islice(self.iter_entries(), limit))

115. class Feed(object):
def iter_entries(self):
start_id = sys.maxint
while True:
entry_ids = exc_sqls("select id from entry where
feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id))
if not entry_ids:
break
for entry_id in entry_ids: =
yield Entry.get(entry_id)
len(self.feeds) * 5 ~
start_id = entry_ids[-1]
len(self.feeds)*5 + limit -5
class FeedCollection(object):
def iter_entries(self):
return imerge(*[feed.iter_entries() for feed in self.feeds])
def get_entries(self, limit=10):
return list(islice(self.iter_entries(), limit))

116. class Feed(object):
def iter_entries(self):
start_id = sys.maxint
while True:
entry_ids = exc_sqls("select id from entry where
feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id))
if not entry_ids:
break
for entry_id in entry_ids:
yield Entry.get(entry_id)
start_id = entry_ids[-1] Entry.get =
0 ~ len(self.feeds)-1
class FeedCollection(object):
def iter_entries(self):
return imerge(*[feed.iter_entries() for feed in self.feeds])
def get_entries(self, limit=10):
return list(islice(self.iter_entries(), limit))

117. decorator generator

118. •

119. class User(object):
def __init__(self, id, username, screen_name, sig):
self.id = id
self.username = username
self.screen_name = screen_name
self.sig = sig
user = User('1002211', 'hongqn', 'hongqn', "
")

120. cPickle vs. marshal
$ python -m timeit -s '
> from user import user
> from cPickle import dumps, loads
> s = dumps(user, 2)'
> 'loads(s)'
100000 loops, best of 3: 6.6 usec per loop
$ python -m timeit -s '
> from user import user
> from marshal import dumps, loads
> d = (user.id, user.username, user.screen_name, user.sig)
> s = dumps(d, 2)' 'loads(s)'
1000000 loops, best of 3: 0.9 usec per loop

121. cPickle vs. marshal
$ python -m timeit -s '
> from user import user
> from cPickle import dumps, loads
> s = dumps(user, 2)'
> 'loads(s)'
7
100000 loops, best of 3: 6.6 usec per loop
$ python -m timeit -s '
> from user import user
> from marshal import dumps, loads
> d = (user.id, user.username, user.screen_name, user.sig)
> s = dumps(d, 2)' 'loads(s)'
1000000 loops, best of 3: 0.9 usec per loop

122. cPickle vs. marshal
$ python -m timeit -s '
> from user import user
> from cPickle import dumps, loads
> s = dumps(user, 2)'
> 'loads(s)'
7
100000 loops, best of 3: 6.6 usec per loop
$ python -m timeit -s '
> from user import user
> from marshal import dumps, loads
> d = (user.id, user.username, user.screen_name, user.sig)
> s = dumps(d, 2)' 'loads(s)'
1000000 loops, best of 3: 0.9 usec per loop

123. cPickle vs. marshal
$ python -c timeit
'
> import cPickle, marshal
> from user import user
> print "pickle:", len(cPickle.dumps(user, 2))
> print "marshal:", len(marshal.dumps((user.id,
> user.username, user.screen_name, user.sig), 2))'
pickle: 129
marshal: 74
43%

124. cPickle vs. marshal
$ python -c timeit
'
> import cPickle, marshal
> from user import user
> print "pickle:", len(cPickle.dumps(user, 2))
> print "marshal:", len(marshal.dumps((user.id,
> user.username, user.screen_name, user.sig), 2))'
pickle: 129
marshal: 74
43%

125. namedtuple
from collections import namedtuple
User = namedtuple('User', 'id username screen_name sig')
user = User('1002211', 'hongqn', 'hongqn', sig="
")
user.username
-> 'hongqn'

126. __metaclass__
class User(tuple):
__metaclass__ = NamedTupleMetaClass
__attrs__ = ['id', 'username', 'screen_name', 'sig']
user = User('1002211', 'hongqn', 'hongqn', sig="
")
s = marshal.dumps(user.__marshal__())
User.__load_marshal__(marshal.loads(s))

127. from operator import itemgetter
class NamedTupleMetaClass(type):
def __new__(mcs, name, bases, dict):
assert bases == (tuple,)
for i, a in enumerate(dict['__attrs__']):
dict[a] = property(itemgetter(i))
dict['__slots__'] = ()
dict['__marshal__'] = tuple
dict['__load_marshal__'] = classmethod(tuple.__new__)
dict['__getnewargs__'] = lambda self: tuple(self)
argtxt = repr(tuple(attrs)).replace("'", "")[1:-1]
template = """def newfunc(cls, %(argtxt)s):
return tuple.__new__(cls, (%(argtxt)s))""" % locals()
namespace = {}
exec template in namespace
dict['__new__'] = namespace['newfunc']
return type.__new__(mcs, name, bases, dict)

128. Warning!

129. • request.get_environ(key)
• e.g.
request.get_environ('REMOTE_ADDR')
--> request.remote_addr

130. descriptor
• __get__, __set__
__delete__
class Descriptor(object):
def __get__(self, instance, owner):
return 'descriptor'
class Owner(object):
attr = Descriptor()
owner = Owner()
owner.attr --> 'descriptor'

131. descriptor
• classmethod
• staticmethod
• property
class C(object):
def get_x(self):
return self._x
def set_x(self, x):
self._x = x
x = property(get_x, set_x)

132. class environ_getter(object):
def __init__(self, key, default=None):
self.key = key
self.default = default
def __get__(self, obj, objtype):
if obj is None:
return self
return obj.get_environ(self.key, self.default)
class HTTPRequest(quixote.http_request.HTTPRequest):
for key in ['HTTP_REFERER', 'REMOTE_ADDR',
'SERVER_NAME', 'REQUEST_URI', 'HTTP_HOST']:
locals()[key.lower()] = environ_getter(key)
locals()
del key

133. class environ_getter(object):
def __init__(self, key, default=None):
self.key = key
self.default = default
def __get__(self, obj, objtype):
if obj is None:
return self
return obj.get_environ(self.key, self.default)
class HTTPRequest(quixote.http_request.HTTPRequest):
for key in ['HTTP_REFERER', 'REMOTE_ADDR',
'SERVER_NAME', 'REQUEST_URI', 'HTTP_HOST']:
locals()[key.lower()] = environ_getter(key)
del key

134. • urllib.urlopen socks

135. Monkey Patch

136. import httplib
orig_connect = httplib.HTTPConnection.connect
def _patched_connect(self):
if HOSTS_BLOCKED.match(self.host):
return _connect_via_socks_proxy(self)
else:
return orig_connect(self)
def _connect_via_socks_proxy(self):
...
httplib.HTTPConnection.connect = _patched_connect

137. Python

138. Python
• Pythonic!

139. Python
• Pythonic!
• Avoid gotchas http://www.ferg.org/projects/python_gotchas.html

140. Python
• Pythonic!
• Avoid gotchashttp://www.ferg.org/projects/python_gotchas.html
• Unicode / Character Encoding

141. Python
• Pythonic!
• Avoid gotchas http://www.ferg.org/projects/python_gotchas.html
• Unicode / Character Encoding
• GIL (Global Interpreter Lock)

142. Python
• Pythonic!
• Avoid gotchas http://www.ferg.org/projects/python_gotchas.html
• Unicode / Character Encoding
• GIL (Global Interpreter Lock)
• Garbage Collection

143. • :Vim / Emacs / Ulipad
• : subversion / mercurial / git
• wiki/ / : Trac
• : Bitten

144. Python
Implementations

145. Python
Implementations
• CPython http://www.python.org/

146. Python
Implementations
• CPython http://www.python.org/
• Unlanden-Swallow http://code.google.com/p/unladen-swallow/

147. Python
Implementations
• CPython http://www.python.org/
• Unlanden-Swallow http://code.google.com/p/unladen-swallow/
• Stackless Python http://www.stackless.com/

148. Python
Implementations
• CPython http://www.python.org/
• Unlanden-Swallow http://code.google.com/p/unladen-swallow/
• Stackless Python http://www.stackless.com/
• IronPython http://ironpython.net/

149. Python
Implementations
• CPython http://www.python.org/
• Unlanden-Swallow http://code.google.com/p/unladen-swallow/
• Stackless Python http://www.stackless.com/
• IronPython http://ironpython.net/
• Jython http://www.jython.org/

150. Python
Implementations
• CPython http://www.python.org/
• Unlanden-Swallow http://code.google.com/p/unladen-swallow/
• Stackless Python http://www.stackless.com/
• IronPython http://ironpython.net/
• Jython http://www.jython.org/
• PyPy http://pypy.org/

151. Q &A