Python                             Web 2.0                               QCon Beijing 2010http://www.flickr.com/photos/arno...
About Me• Python• 2002             Python• 2004  Python• http://www.douban.com/  people/hongqn/• hongqn@douban.com• http:/...
Python•   Python is a programming language that lets you    work more quickly and integrate your systems more    effective...
Languages in              C             27%                   Javascript                      12%                         ...
Why Python?
• Hello World: 1
• Hello World: 1•            :1
• Hello World: 1•            :1•         :1
• Hello World: 1•            :1•         :1•         :3
: 13
: 13import osfrom collections import defaultdictd = defaultdict(int)for dirpath, dirnames, filenames in os.walk(.):    for...
•• Pythonic
•    1. svn ci    2. svn up    3. restart
• Web•••
• Battery Included:             200+• PyPI: 9613 packages currently• /            /    /     /         /   /          /......
Python
Python
PythonJust kidding :-p
Web Server
Web Server• python -m SimpleHTTPServer
Web Server• python -m SimpleHTTPServer
web.py                http://webpy.org/import weburls = (    /(.*), hello)app = web.application(urls, globals())class hell...
Flask       http://flask.pocoo.org/import flask import Flaskapp = Flask(__name__)@app.route("/<name>")def hello(name):    i...
WSGI   http://www.python.org/dev/peps/pep-0333/
Why so many Python  web frameworks?• Because you can write your own  framework in 3 hours and a total of 60  lines of Pyth...
doctestdef cube(x):    """    >>> cube(10)    1000    """    return x * xdef _test():    import doctest    doctest.testmod...
nose    http://somethingaboutorange.com/mrl/projects/nose/from cube import cubedef test_cube():    result = cube(10)    as...
numpy           http://numpy.scipy.org/>>> from numpy import *>>> A = arange(4).reshape(2, 2)>>> Aarray([[0, 1],       [2,...
ipython          http://numpy.scipy.org/$ ipython -pylabIn [1]: X = frange(0, 10, 0.1)In [2]: Y = [sin(x) for x in X]In [3...
ipython          http://numpy.scipy.org/$ ipython -pylabIn [1]: X = frange(0, 10, 0.1)In [2]: Y = [sin(x) for x in X]In [3...
virtualenv            http://virtualenv.openplans.org/                                python$ python go-pylons.py --no-sit...
Pyrex/Cythoncdef extern from "math.h"    double sin(double)cdef double f(double x):    return sin(x*x)
Pythonic
>>> import thisThe Zen of Python, by Tim Peters         http://bit.ly/pyzencnBeautiful is better than ugly.Explicit is bet...
In the face of ambiguity, refuse thetemptation to guess.There should be one-- andpreferably only one --obvious wayto do it...
Simple is better than       complexclass HelloWorld{        public static void main(String args[])        {           Syst...
Simple is better than      complex    print "Hello World!"
Readability counts
Readability counts•           {}   end
Readability counts•                  {}   end•                 (except "@" for    decorators)
Readability counts•                    {}   end•                  (except "@" for    decorators)if limit is not None and l...
TOOWTDI• There (should be) Only One Way To Do It.• vs. Perlish TIMTOWTDI (There Is More  Than One Way To Do It)
TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More    Than One Way To Do It)a = [1,...
TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More    Than One Way To Do It)a = [1,...
TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More    Than One Way To Do It)a = [1,...
TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More    Than One Way To Do It)a = [1,...
TOOWTDI• There (should be) Only One Way To Do It.• vs. Perlish TIMTOWTDI (There Is More  Than One Way To Do It)     b = [x...
http://twitter.com/robbinfan/status/9879724095http://twitter.com/hongqn/status/9883515681
Python                                                            Chttp://www.flickr.com/photos/nicksieger/281055485/   htt...
Rubyhttp://www.flickr.com/photos/nicksieger/280661836/
Javahttp://www.flickr.com/photos/nicksieger/280662707/
Python
• svn
• svn•       (   list)
• svn•           (   list)•       +
• svn•                    (   list)•       +•           parser
config.pyMEMCACHED_ADDR = [localhost:11211]from local_config import *
config.pyMEMCACHED_ADDR = [localhost:11211]from local_config import *                                       local_config.py ...
config.pyMEMCACHED_ADDR = [localhost:11211]from local_config import *                    .py                local_config.py ...
•
class GroupUI(object):    def new_topic(self, request):        if self.group.can_post(request.user):            return new...
class GroupUI(object):    @check_permission(post, msg="               ")    def new_topic(self, request):        return ne...
decoratordef print_before_exec(func):    def _(*args, **kwargs):        print "decorated"        return func(*args, **kwar...
decoratordef print_before_exec(func):    def _(*args, **kwargs):        print "decorated"        return func(*args, **kwar...
class check_permission(object):    def __init__(self, action, msg=None):        self.action = action        self.msg = msg...
class check_permission(object):    def __init__(self, action, msg=None):        self.action = action        self.msg = msg...
class check_permission(object):    def __init__(self, action, msg=None):        self.action = action        self.msg = msg...
class check_permission(object):    def __init__(self, action, msg=None):        self.action = action        self.msg = msg...
class check_permission(object):    def __init__(self, action, msg=None):        self.action = action        self.msg = msg...
class check_permission(object):    def __init__(self, action, msg=None):        self.action = action        self.msg = msg...
class GroupUI(object):    @check_permission(post, msg="               ")    def new_topic(self, request):        return ne...
•
def send_notification_mail(email, subject, body):    msg = MSG_SEND_MAIL + 0 + email + 0 + subject + 0 + body    mq.put(ms...
@asyncdef send_notification_mail(email, subject, body):    fromaddr = no-reply@douban.com    email_body = make_email_body(...
def async(func):    mod = sys.modules[func.__module__]    fname = origin_ + func.__name__    mod[fname] = func    def _(*a...
def async(func):    mod = sys.modules[func.__module__]    fname = origin_ + func.__name__    mod[fname] = func    def _(*a...
def async(func):    mod = sys.modules[func.__module__]    fname = origin_ + func.__name__    mod[fname] = func    def _(*a...
def async(func):    mod = sys.modules[func.__module__]    fname = origin_ + func.__name__    mod[fname] = func    def _(*a...
def async(func):    mod = sys.modules[func.__module__]    fname = origin_ + func.__name__    mod[fname] = func    def _(*a...
def async(func):    mod = sys.modules[func.__module__]    fname = origin_ + func.__name__    mod[fname] = func    def _(*a...
def async(func):    mod = sys.modules[func.__module__]    fname = origin_ + func.__name__    mod[fname] = func    def _(*a...
def async(func):    mod = sys.modules[func.__module__]    fname = origin_ + func.__name__    mod[fname] = func    def _(*a...
• cache   (SQL,   , etc)
def get_latest_review_id():    review_id = mc.get(latest_review_id)    if review_id is None:        review_id = exc_sql("s...
@cache(latest_review_id)def get_latest_review_id():    return exc_sql("select max(id) from review")
def cache(key):    def deco(func):        def _(*args, **kwargs):            r = mc.get(key)            if r is None:     ...
def cache(key):    def deco(func):        def _(*args, **kwargs):            r = mc.get(key)            if r is None:     ...
cache keydef get_review(id):    key = review:%s % id    review = mc.get(key)    if review is None: # cache miss        id,...
cache key                       decorator@cache(review:{id})def get_review(id):    id, author_id, text = exc_sql("select i...
def cache(key_pattern, expire=0):    def deco(f):        arg_names, varargs, varkw, defaults = inspect.getargspec(f)      ...
def cache(key_pattern, expire=0):    def deco(f):        arg_names, varargs, varkw, defaults = inspect.getargspec(f)      ...
def cache(key_pattern, expire=0):    def deco(f):        arg_names, varargs, varkw, defaults = inspect.getargspec(f)      ...
inspect.getargspec>>> import inspect>>> def f(a, b=1, c=2):...     pass...>>> inspect.getargspec(f)ArgSpec(args=[a, b, c],...
def cache(key_pattern, expire=0):    def deco(f):        arg_names, varargs, varkw, defaults = inspect.getargspec(f)      ...
def cache(key_pattern, expire=0):    def deco(f):        arg_names, varargs, varkw, defaults = inspect.getargspec(f)      ...
def cache(key_pattern, expire=0):    def deco(f):        arg_names, varargs, varkw, defaults = inspect.getargspec(f)      ...
def cache(key_pattern, expire=0):    def deco(f):        arg_names, varargs, varkw, defaults = inspect.getargspec(f)      ...
• feed       feed  entry_id
class Feed(object):    def get_entries(self, limit=10):        ids = exc_sqls("select id from entry where feed_id=%s order...
class Feed(object):    def get_entries(self, limit=10):        ids = exc_sqls("select id from entry where feed_id=%s order...
class Feed(object):    def get_entries(self, limit=10):        ids = exc_sqls("select id from entry where feed_id=%s order...
class Feed(object):    def get_entries(self, limit=10):        ids = exc_sqls("select id from entry where feed_id=%s order...
class Feed(object):    def get_entries(self, limit=10):        ids = exc_sqls("select id from entry where feed_id=%s order...
iterator and generatordef fib():    x, y = 1, 1    while True:        yield x        x, y = y, x+ydef odd(seq):    return ...
itertools•   count([n]) --> n, n+1, n+2•   cycle(p) --> p0, p1, ... plast, p0, p1, ...•   repeat(elem [,n]) --> elem, elem...
class Feed(object):    def iter_entries(self):        start_id = sys.maxint        while True:            entry_ids = exc_...
class Feed(object):    def iter_entries(self):        start_id = sys.maxint        while True:            entry_ids = exc_...
class Feed(object):    def iter_entries(self):        start_id = sys.maxint        while True:            entry_ids = exc_...
class Feed(object):    def iter_entries(self):        start_id = sys.maxint        while True:            entry_ids = exc_...
class Feed(object):    def iter_entries(self):        start_id = sys.maxint        while True:            entry_ids = exc_...
class Feed(object):    def iter_entries(self):        start_id = sys.maxint        while True:            entry_ids = exc_...
class Feed(object):    def iter_entries(self):        start_id = sys.maxint        while True:            entry_ids = exc_...
decorator   generator
•
class User(object):    def __init__(self, id, username, screen_name, sig):        self.id = id        self.username = user...
cPickle vs. marshal$ python -m timeit -s > from user import user> from cPickle import dumps, loads> s = dumps(user, 2) > l...
cPickle vs. marshal$ python -m timeit -s > from user import user> from cPickle import dumps, loads> s = dumps(user, 2) > l...
cPickle vs. marshal$ python -m timeit -s > from user import user> from cPickle import dumps, loads> s = dumps(user, 2) > l...
cPickle vs. marshal$ python -c timeit             > import cPickle, marshal> from user import user> print "pickle:", len(c...
cPickle vs. marshal$ python -c timeit             > import cPickle, marshal> from user import user> print "pickle:", len(c...
namedtuplefrom collections import namedtupleUser = namedtuple(User, id username screen_name sig)user = User(1002211, hongq...
__metaclass__class User(tuple):    __metaclass__ = NamedTupleMetaClass    __attrs__ = [id, username, screen_name, sig]user...
from operator import itemgetterclass NamedTupleMetaClass(type):    def __new__(mcs, name, bases, dict):        assert base...
Warning!
•        request.get_environ(key)    • e.g.      request.get_environ(REMOTE_ADDR)      --> request.remote_addr
descriptor•            __get__, __set__    __delete__    class Descriptor(object):        def __get__(self, instance, owne...
descriptor• classmethod• staticmethod• property     class C(object):         def get_x(self):             return self._x  ...
class environ_getter(object):    def __init__(self, key, default=None):        self.key = key        self.default = defaul...
class environ_getter(object):    def __init__(self, key, default=None):        self.key = key        self.default = defaul...
•   urllib.urlopen   socks
Monkey Patch
import httpliborig_connect = httplib.HTTPConnection.connectdef _patched_connect(self):    if HOSTS_BLOCKED.match(self.host...
Python
Python• Pythonic!
Python• Pythonic!• Avoid gotchas   http://www.ferg.org/projects/python_gotchas.html
Python• Pythonic!• Avoid gotchashttp://www.ferg.org/projects/python_gotchas.html• Unicode / Character Encoding
Python• Pythonic!• Avoid gotchas http://www.ferg.org/projects/python_gotchas.html• Unicode / Character Encoding• GIL (Glob...
Python• Pythonic!• Avoid gotchas http://www.ferg.org/projects/python_gotchas.html• Unicode / Character Encoding• GIL (Glob...
•         :Vim / Emacs / Ulipad•           : subversion / mercurial / git• wiki/          /          : Trac•           : B...
PythonImplementations
Python    Implementations• CPython   http://www.python.org/
Python     Implementations• CPython http://www.python.org/• Unlanden-Swallow      http://code.google.com/p/unladen-swallow/
Python     Implementations• CPython http://www.python.org/• Unlanden-Swallow      http://code.google.com/p/unladen-swallow...
Python     Implementations• CPython http://www.python.org/• Unlanden-Swallow      http://code.google.com/p/unladen-swallow...
Python       Implementations• CPython     http://www.python.org/• Unlanden-Swallow          http://code.google.com/p/unlad...
Python       Implementations• CPython     http://www.python.org/• Unlanden-Swallow          http://code.google.com/p/unlad...
Q &A
Python 于Web 2.0网站的应用 (QCon Beijing 2010)
Python 于Web 2.0网站的应用 (QCon Beijing 2010)
Python 于Web 2.0网站的应用 (QCon Beijing 2010)
Python 于Web 2.0网站的应用 (QCon Beijing 2010)
Upcoming SlideShare
Loading in …5
×

Python 于Web 2.0网站的应用 (QCon Beijing 2010)

953 views

Published on

Published in: Technology
1 Comment
3 Likes
Statistics
Notes
  • 案例四的两种实现结果是不是不一样?
    实现1会把self.feeds * limit个entry按entry.id倒序排列
    实现2是依此取各个self.feeds的iterator的第n个
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
953
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
13
Comments
1
Likes
3
Embeds 0
No embeds

No notes for slide

Python 于Web 2.0网站的应用 (QCon Beijing 2010)

  1. 1. Python Web 2.0 QCon Beijing 2010http://www.flickr.com/photos/arnolouise/2986467632/
  2. 2. About Me• Python• 2002 Python• 2004 Python• http://www.douban.com/ people/hongqn/• hongqn@douban.com• http://twitter.com/hongqn
  3. 3. Python• Python is a programming language that lets you work more quickly and integrate your systems more effectively. You can learn to use Python and see almost immediate gains in productivity and lower maintenance costs. (via http://python.org/)
  4. 4. Languages in C 27% Javascript 12% C++ Python 3% 58% (Pyrex/R/Erlang/Go/Shell) 1%
  5. 5. Why Python?
  6. 6. • Hello World: 1
  7. 7. • Hello World: 1• :1
  8. 8. • Hello World: 1• :1• :1
  9. 9. • Hello World: 1• :1• :1• :3
  10. 10. : 13
  11. 11. : 13import osfrom collections import defaultdictd = defaultdict(int)for dirpath, dirnames, filenames in os.walk(.): for filename in filenames: path = os.path.join(dirpath, filename) ext = os.path.splitext(filename)[1] d[ext] += len(list(open(path)))for ext, n_lines in d.items(): print ext, n_lines
  12. 12. •• Pythonic
  13. 13. • 1. svn ci 2. svn up 3. restart
  14. 14. • Web•••
  15. 15. • Battery Included: 200+• PyPI: 9613 packages currently• / / / / / / /...• easily extensible
  16. 16. Python
  17. 17. Python
  18. 18. PythonJust kidding :-p
  19. 19. Web Server
  20. 20. Web Server• python -m SimpleHTTPServer
  21. 21. Web Server• python -m SimpleHTTPServer
  22. 22. web.py http://webpy.org/import weburls = ( /(.*), hello)app = web.application(urls, globals())class hello: def GET(self, name): if not name: name = World return Hello, + name + !if __name__ == "__main__": app.run()
  23. 23. Flask http://flask.pocoo.org/import flask import Flaskapp = Flask(__name__)@app.route("/<name>")def hello(name): if not name: name = World return Hello, + name + !if __name__ == "__main__": app.run()
  24. 24. WSGI http://www.python.org/dev/peps/pep-0333/
  25. 25. Why so many Python web frameworks?• Because you can write your own framework in 3 hours and a total of 60 lines of Python code.• http://bitworking.org/news/ Why_so_many_Python_web_frameworks
  26. 26. doctestdef cube(x): """ >>> cube(10) 1000 """ return x * xdef _test(): import doctest doctest.testmod()if __name__ == "__main__": _test()
  27. 27. nose http://somethingaboutorange.com/mrl/projects/nose/from cube import cubedef test_cube(): result = cube(10) assert result == 1000
  28. 28. numpy http://numpy.scipy.org/>>> from numpy import *>>> A = arange(4).reshape(2, 2)>>> Aarray([[0, 1], [2, 3]])>>> dot(A, A.T)array([[ 1, 3], [ 3, 13]])
  29. 29. ipython http://numpy.scipy.org/$ ipython -pylabIn [1]: X = frange(0, 10, 0.1)In [2]: Y = [sin(x) for x in X]In [3]: plot(X, Y)
  30. 30. ipython http://numpy.scipy.org/$ ipython -pylabIn [1]: X = frange(0, 10, 0.1)In [2]: Y = [sin(x) for x in X]In [3]: plot(X, Y)
  31. 31. virtualenv http://virtualenv.openplans.org/ python$ python go-pylons.py --no-site-packages mydevenv$ cd mydevenv$ source bin/activate(mydevenv)$ paster create -t new9 helloworld
  32. 32. Pyrex/Cythoncdef extern from "math.h" double sin(double)cdef double f(double x): return sin(x*x)
  33. 33. Pythonic
  34. 34. >>> import thisThe Zen of Python, by Tim Peters http://bit.ly/pyzencnBeautiful is better than ugly.Explicit is better than implicit.Simple is better than complex.Complex is better than complicated.Flat is better than nested.Sparse is better than dense.Readability counts.Special cases arent special enough tobreak the rules.  Although practicality beats purity.Errors should never pass silently.Unless explicitly silenced.  
  35. 35. In the face of ambiguity, refuse thetemptation to guess.There should be one-- andpreferably only one --obvious wayto do it.Although that way may not be Pythonobvious at first unless youre Dutch.  Now is better than never.Although never is often better than*right* now.  If the implementation is hard toexplain, its a bad idea.If the implementation is easy to  explain, it may be a good idea.Namespaces are one honking greatidea -- lets do more of those!
  36. 36. Simple is better than complexclass HelloWorld{ public static void main(String args[]) { System.out.println("Hello World!"); }}
  37. 37. Simple is better than complex print "Hello World!"
  38. 38. Readability counts
  39. 39. Readability counts• {} end
  40. 40. Readability counts• {} end• (except "@" for decorators)
  41. 41. Readability counts• {} end• (except "@" for decorators)if limit is not None and len(ids)>limit: ids = random.sample(ids, limit)
  42. 42. TOOWTDI• There (should be) Only One Way To Do It.• vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It)
  43. 43. TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It)a = [1, 2, 3, 4, 5]b = []for i in range(len(a)): b.append(a[i]*2)
  44. 44. TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It)a = [1, 2, 3, 4, 5]b = []for i in range(len(a)): b.append(a[i]*2)
  45. 45. TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It)a = [1, 2, 3, 4, 5] b = []b = [] for x in a:for i in range(len(a)): b.append(x*2) b.append(a[i]*2)
  46. 46. TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It)a = [1, 2, 3, 4, 5] b = []b = [] for x in a:for i in range(len(a)): b.append(x*2) b.append(a[i]*2)
  47. 47. TOOWTDI• There (should be) Only One Way To Do It.• vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It) b = [x*2 for x in a]
  48. 48. http://twitter.com/robbinfan/status/9879724095http://twitter.com/hongqn/status/9883515681
  49. 49. Python Chttp://www.flickr.com/photos/nicksieger/281055485/ http://www.flickr.com/photos/nicksieger/281055530/
  50. 50. Rubyhttp://www.flickr.com/photos/nicksieger/280661836/
  51. 51. Javahttp://www.flickr.com/photos/nicksieger/280662707/
  52. 52. Python
  53. 53. • svn
  54. 54. • svn• ( list)
  55. 55. • svn• ( list)• +
  56. 56. • svn• ( list)• +• parser
  57. 57. config.pyMEMCACHED_ADDR = [localhost:11211]from local_config import *
  58. 58. config.pyMEMCACHED_ADDR = [localhost:11211]from local_config import * local_config.py MEMCACHED_ADDR = [ frodo:11211, sam:11211, pippin:11211, merry:11211, ]
  59. 59. config.pyMEMCACHED_ADDR = [localhost:11211]from local_config import * .py local_config.py exec MEMCACHED_ADDR = [ frodo:11211, sam:11211, pippin:11211, merry:11211, ]
  60. 60.
  61. 61. class GroupUI(object): def new_topic(self, request): if self.group.can_post(request.user): return new_topic_ui(self.group) else: request.response.set_status(403, "Forbidden") return error_403_ui(msg=" ") def join(self, request): if self.group.can_join(request.user): ...class Group(object): def can_post(self, user): return self.group.has_member(user) def can_join(self, user): return not self.group.has_banned(user)
  62. 62. class GroupUI(object): @check_permission(post, msg=" ") def new_topic(self, request): return new_topic_ui(self.group) @check_permission(join, msg=" ") def join(self, request): ...class Group(object): def can_post(self, user): return self.group.has_member(user) def can_join(self, user): return not self.group.has_banned(user)
  63. 63. decoratordef print_before_exec(func): def _(*args, **kwargs): print "decorated" return func(*args, **kwargs) return _@print_before_execdef double(x): print x*2double(10)
  64. 64. decoratordef print_before_exec(func): def _(*args, **kwargs): print "decorated" return func(*args, **kwargs) return _ decorated@print_before_execdef double(x): 20 print x*2double(10)
  65. 65. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, *args, **kwargs): f = getattr(ui.perm_obj, can_ + action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
  66. 66. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, *args, **kwargs): f = getattr(ui.perm_obj, can_ + action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
  67. 67. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, *args, **kwargs): f = getattr(ui.perm_obj, can_ + action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
  68. 68. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, *args, **kwargs): f = getattr(ui.perm_obj, can_ + action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
  69. 69. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, *args, **kwargs): f = getattr(ui.perm_obj, can_ + action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
  70. 70. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, *args, **kwargs): f = getattr(ui.perm_obj, can_ + action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
  71. 71. class GroupUI(object): @check_permission(post, msg=" ") def new_topic(self, request): return new_topic_ui(self.group) @check_permission(join, msg=" ") def join(self, request): ...class Group(object): def can_post(self, user): return self.group.has_member(user) def can_join(self, user): return not self.group.has_banned(user)
  72. 72.
  73. 73. def send_notification_mail(email, subject, body): msg = MSG_SEND_MAIL + 0 + email + 0 + subject + 0 + body mq.put(msg)def async_worker(): msg = mq.get() msg = msg.split(0) cmd = msg[0] if cmd == MSG_SEND_MAIL: email, subject, body = msg[1:] fromaddr = no-reply@douban.com email_body = make_email_body(fromaddr, email, subject, body) smtp = smtplib.SMTP(mail) smtp.sendmail(fromaddr, email, email_body) elif cmd == MSG_xxxx: ... elif cmd == MSG_yyyy: ...
  74. 74. @asyncdef send_notification_mail(email, subject, body): fromaddr = no-reply@douban.com email_body = make_email_body(fromaddr, email, subject,body) smtp = smtplib.SMTP(mail) smtp.sendmail(fromaddr, email, email_body)
  75. 75. def async(func): mod = sys.modules[func.__module__] fname = origin_ + func.__name__ mod[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod[fname](*a, **kw)
  76. 76. def async(func): mod = sys.modules[func.__module__] fname = origin_ + func.__name__ mod[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod[fname](*a, **kw)
  77. 77. def async(func): mod = sys.modules[func.__module__] fname = origin_ + func.__name__ mod[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod[fname](*a, **kw)
  78. 78. def async(func): mod = sys.modules[func.__module__] fname = origin_ + func.__name__ mod[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod[fname](*a, **kw)
  79. 79. def async(func): mod = sys.modules[func.__module__] fname = origin_ + func.__name__ mod[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod[fname](*a, **kw)
  80. 80. def async(func): mod = sys.modules[func.__module__] fname = origin_ + func.__name__ mod[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod[fname](*a, **kw)
  81. 81. def async(func): mod = sys.modules[func.__module__] fname = origin_ + func.__name__ mod[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod[fname](*a, **kw)
  82. 82. def async(func): mod = sys.modules[func.__module__] fname = origin_ + func.__name__ mod[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod[fname](*a, **kw)
  83. 83. • cache (SQL, , etc)
  84. 84. def get_latest_review_id(): review_id = mc.get(latest_review_id) if review_id is None: review_id = exc_sql("select max(id) from review") mc.set(latest_review_id, review_id) return review_id
  85. 85. @cache(latest_review_id)def get_latest_review_id(): return exc_sql("select max(id) from review")
  86. 86. def cache(key): def deco(func): def _(*args, **kwargs): r = mc.get(key) if r is None: r = func(*args, **kwargs) mc.set(key, r) return r return _ return deco
  87. 87. def cache(key): def deco(func): def _(*args, **kwargs): r = mc.get(key) if r is None: r = func(*args, **kwargs) mc.set(key, r) return r return _ return deco
  88. 88. cache keydef get_review(id): key = review:%s % id review = mc.get(key) if review is None: # cache miss id, author_id, text = exc_sql("select id, author_id,text from review where id=%s", id) review = Review(id, author_id, text) mc.set(key, review) return review
  89. 89. cache key decorator@cache(review:{id})def get_review(id): id, author_id, text = exc_sql("select id, author_id,text from review where id=%s", id) return Review(id, author_id, text)
  90. 90. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: r = f(*a, **kw) mc.set(key, r, expire) return r return _ return deco
  91. 91. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: r = f(*a, **kw) mc.set(key, r, expire) return r return _ return deco
  92. 92. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: r = f(*a, **kw) mc.set(key, r, expire) return r return _ return deco
  93. 93. inspect.getargspec>>> import inspect>>> def f(a, b=1, c=2):... pass...>>> inspect.getargspec(f)ArgSpec(args=[a, b, c], varargs=None, keywords=None,defaults=(1, 2))>>>>>>>>> def f(a, b=1, c=2, *args, **kwargs):... pass...>>> inspect.getargspec(f)ArgSpec(args=[a, b, c], varargs=args, keywords=kwargs,defaults=(1, 2))
  94. 94. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: r = f(*a, **kw) mc.set(key, r, expire) return r return _ return deco
  95. 95. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: hint: r = f(*a, **kw) mc.set(key, r, expire) return r • str.format in python 2.6: {id}.format(id=1) => 1 return _ return deco • dict(zip([a, b, c], [1, 2, 3])) => {a: 1, b: 2, c: 3}
  96. 96. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: hint: r = f(*a, **kw) mc.set(key, r, expire) return r • str.format in python 2.6: {id}.format(id=1) => 1 return _ return deco • dict(zip([a, b, c], [1, 2, 3])) => {a: 1, b: 2, c: 3}
  97. 97. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: r = f(*a, **kw) mc.set(key, r, expire) return r return _ return deco
  98. 98. • feed feed entry_id
  99. 99. class Feed(object): def get_entries(self, limit=10): ids = exc_sqls("select id from entry where feed_id=%s order by id desc limit %s", (self.id, limit)) return [Entry.get(id) for id in ids]class FeedCollection(object): def get_entries(self, limit=10): mixed_entries = [] for feed in self.feeds: entries = feed.get_entries(limit=limit) mixed_entries += entries mixed_entries.sort(key=lambda e: e.id, reverse=True) return mixed_entries[:10]
  100. 100. class Feed(object): def get_entries(self, limit=10): ids = exc_sqls("select id from entry where feed_id=%s order by id desc limit %s", (self.id, limit)) return [Entry.get(id) for id in ids]class FeedCollection(object): def get_entries(self, limit=10): mixed_entries = [] for feed in self.feeds: entries = feed.get_entries(limit=limit) mixed_entries += entries mixed_entries.sort(key=lambda e: e.id, reverse=True) return mixed_entries[:10]
  101. 101. class Feed(object): def get_entries(self, limit=10): ids = exc_sqls("select id from entry where feed_id=%s order by id desc limit %s", (self.id, limit)) return [Entry.get(id) for id in ids]class FeedCollection(object): def get_entries(self, limit=10): mixed_entries = [] for feed in self.feeds: entries = feed.get_entries(limit=limit) mixed_entries += entries mixed_entries.sort(key=lambda e: e.id, reverse=True) return mixed_entries[:10]
  102. 102. class Feed(object): def get_entries(self, limit=10): ids = exc_sqls("select id from entry where feed_id=%s order by id desc limit %s", (self.id, limit)) return [Entry.get(id) for id in ids]class FeedCollection(object): = def get_entries(self, limit=10): len(self.feeds) * limit mixed_entries = [] for feed in self.feeds: entries = feed.get_entries(limit=limit) mixed_entries += entries mixed_entries.sort(key=lambda e: e.id, reverse=True) return mixed_entries[:10]
  103. 103. class Feed(object): def get_entries(self, limit=10): ids = exc_sqls("select id from entry where feed_id=%s order by id desc limit %s", (self.id, limit)) return [Entry.get(id) for id in ids]class FeedCollection(object): Entry.get = def get_entries(self, limit=10): len(self.feeds-1) * limit mixed_entries = [] for feed in self.feeds: entries = feed.get_entries(limit=limit) mixed_entries += entries mixed_entries.sort(key=lambda e: e.id, reverse=True) return mixed_entries[:10]
  104. 104. iterator and generatordef fib(): x, y = 1, 1 while True: yield x x, y = y, x+ydef odd(seq): return (n for n in seq if n%2)def less_than(seq, upper_limit): for number in seq: if number >= upper_limit: break yield numberprint sum(odd(less_than(fib(), 4000000)))
  105. 105. itertools• count([n]) --> n, n+1, n+2• cycle(p) --> p0, p1, ... plast, p0, p1, ...• repeat(elem [,n]) --> elem, elem, elem, ... endless or up to n times• izip(p, q, ...) --> (p[0], q[0]), (p[1], q[1]), ...• islice(seq, [start,] stop [, step]) --> elements from seq[start:stop:step]• ... and more ...
  106. 106. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry wherefeed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1]class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
  107. 107. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry wherefeed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1]class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
  108. 108. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry wherefeed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1]class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
  109. 109. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry wherefeed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1]class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
  110. 110. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry wherefeed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1]class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
  111. 111. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry wherefeed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: = yield Entry.get(entry_id) len(self.feeds) * 5 ~ start_id = entry_ids[-1] len(self.feeds)*5 + limit -5class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
  112. 112. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry wherefeed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1] Entry.get =0class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
  113. 113. decorator generator
  114. 114.
  115. 115. class User(object): def __init__(self, id, username, screen_name, sig): self.id = id self.username = username self.screen_name = screen_name self.sig = siguser = User(1002211, hongqn, hongqn, " ")
  116. 116. cPickle vs. marshal$ python -m timeit -s > from user import user> from cPickle import dumps, loads> s = dumps(user, 2) > loads(s)100000 loops, best of 3: 6.6 usec per loop$ python -m timeit -s > from user import user> from marshal import dumps, loads> d = (user.id, user.username, user.screen_name, user.sig)> s = dumps(d, 2) loads(s)1000000 loops, best of 3: 0.9 usec per loop
  117. 117. cPickle vs. marshal$ python -m timeit -s > from user import user> from cPickle import dumps, loads> s = dumps(user, 2) > loads(s) 7100000 loops, best of 3: 6.6 usec per loop$ python -m timeit -s > from user import user> from marshal import dumps, loads> d = (user.id, user.username, user.screen_name, user.sig)> s = dumps(d, 2) loads(s)1000000 loops, best of 3: 0.9 usec per loop
  118. 118. cPickle vs. marshal$ python -m timeit -s > from user import user> from cPickle import dumps, loads> s = dumps(user, 2) > loads(s) 7100000 loops, best of 3: 6.6 usec per loop$ python -m timeit -s > from user import user> from marshal import dumps, loads> d = (user.id, user.username, user.screen_name, user.sig)> s = dumps(d, 2) loads(s)1000000 loops, best of 3: 0.9 usec per loop
  119. 119. cPickle vs. marshal$ python -c timeit > import cPickle, marshal> from user import user> print "pickle:", len(cPickle.dumps(user, 2))> print "marshal:", len(marshal.dumps((user.id, > user.username, user.screen_name, user.sig), 2))pickle: 129marshal: 74 43%
  120. 120. cPickle vs. marshal$ python -c timeit > import cPickle, marshal> from user import user> print "pickle:", len(cPickle.dumps(user, 2))> print "marshal:", len(marshal.dumps((user.id, > user.username, user.screen_name, user.sig), 2))pickle: 129marshal: 74 43%
  121. 121. namedtuplefrom collections import namedtupleUser = namedtuple(User, id username screen_name sig)user = User(1002211, hongqn, hongqn, sig=" ")user.username-> hongqn
  122. 122. __metaclass__class User(tuple): __metaclass__ = NamedTupleMetaClass __attrs__ = [id, username, screen_name, sig]user = User(1002211, hongqn, hongqn, sig=" ")s = marshal.dumps(user.__marshal__())User.__load_marshal__(marshal.loads(s))
  123. 123. from operator import itemgetterclass NamedTupleMetaClass(type): def __new__(mcs, name, bases, dict): assert bases == (tuple,) for i, a in enumerate(dict[__attrs__]): dict[a] = property(itemgetter(i)) dict[__slots__] = () dict[__marshal__] = tuple dict[__load_marshal__] = classmethod(tuple.__new__) dict[__getnewargs__] = lambda self: tuple(self) argtxt = repr(tuple(attrs)).replace("", "")[1:-1] template = """def newfunc(cls, %(argtxt)s): return tuple.__new__(cls, (%(argtxt)s))""" % locals() namespace = {} exec template in namespace dict[__new__] = namespace[newfunc] return type.__new__(mcs, name, bases, dict)
  124. 124. Warning!
  125. 125. • request.get_environ(key) • e.g. request.get_environ(REMOTE_ADDR) --> request.remote_addr
  126. 126. descriptor• __get__, __set__ __delete__ class Descriptor(object): def __get__(self, instance, owner): return descriptor class Owner(object): attr = Descriptor() owner = Owner() owner.attr --> descriptor
  127. 127. descriptor• classmethod• staticmethod• property class C(object): def get_x(self): return self._x def set_x(self, x): self._x = x x = property(get_x, set_x)
  128. 128. class environ_getter(object): def __init__(self, key, default=None): self.key = key self.default = default def __get__(self, obj, objtype): if obj is None: return self return obj.get_environ(self.key, self.default)class HTTPRequest(quixote.http_request.HTTPRequest): for key in [HTTP_REFERER, REMOTE_ADDR, SERVER_NAME, REQUEST_URI, HTTP_HOST]: locals()[key.lower()] = environ_getter(key) locals() del key
  129. 129. class environ_getter(object): def __init__(self, key, default=None): self.key = key self.default = default def __get__(self, obj, objtype): if obj is None: return self return obj.get_environ(self.key, self.default)class HTTPRequest(quixote.http_request.HTTPRequest): for key in [HTTP_REFERER, REMOTE_ADDR, SERVER_NAME, REQUEST_URI, HTTP_HOST]: locals()[key.lower()] = environ_getter(key) del key
  130. 130. • urllib.urlopen socks
  131. 131. Monkey Patch
  132. 132. import httpliborig_connect = httplib.HTTPConnection.connectdef _patched_connect(self): if HOSTS_BLOCKED.match(self.host): return _connect_via_socks_proxy(self) else: return orig_connect(self)def _connect_via_socks_proxy(self): ...httplib.HTTPConnection.connect = _patched_connect
  133. 133. Python
  134. 134. Python• Pythonic!
  135. 135. Python• Pythonic!• Avoid gotchas http://www.ferg.org/projects/python_gotchas.html
  136. 136. Python• Pythonic!• Avoid gotchashttp://www.ferg.org/projects/python_gotchas.html• Unicode / Character Encoding
  137. 137. Python• Pythonic!• Avoid gotchas http://www.ferg.org/projects/python_gotchas.html• Unicode / Character Encoding• GIL (Global Interpreter Lock)
  138. 138. Python• Pythonic!• Avoid gotchas http://www.ferg.org/projects/python_gotchas.html• Unicode / Character Encoding• GIL (Global Interpreter Lock)• Garbage Collection
  139. 139. • :Vim / Emacs / Ulipad• : subversion / mercurial / git• wiki/ / : Trac• : Bitten
  140. 140. PythonImplementations
  141. 141. Python Implementations• CPython http://www.python.org/
  142. 142. Python Implementations• CPython http://www.python.org/• Unlanden-Swallow http://code.google.com/p/unladen-swallow/
  143. 143. Python Implementations• CPython http://www.python.org/• Unlanden-Swallow http://code.google.com/p/unladen-swallow/• Stackless Python http://www.stackless.com/
  144. 144. Python Implementations• CPython http://www.python.org/• Unlanden-Swallow http://code.google.com/p/unladen-swallow/• Stackless Python http://www.stackless.com/• IronPython http://ironpython.net/
  145. 145. Python Implementations• CPython http://www.python.org/• Unlanden-Swallow http://code.google.com/p/unladen-swallow/• Stackless Python http://www.stackless.com/• IronPython http://ironpython.net/• Jython http://www.jython.org/
  146. 146. Python Implementations• CPython http://www.python.org/• Unlanden-Swallow http://code.google.com/p/unladen-swallow/• Stackless Python http://www.stackless.com/• IronPython http://ironpython.net/• Jython http://www.jython.org/• PyPy http://pypy.org/
  147. 147. Q &A

×