SlideShare a Scribd company logo
1 of 151
Download to read offline
Python
                             Web 2.0



                               QCon Beijing 2010

http://www.flickr.com/photos/arnolouise/2986467632/
About Me
• Python
• 2002             Python
• 2004
  Python
• http://www.douban.com/
  people/hongqn/
• hongqn@douban.com
• http://twitter.com/hongqn
Python
•   Python is a programming language that lets you
    work more quickly and integrate your systems more
    effectively. You can learn to use Python and see
    almost immediate gains in productivity and lower
    maintenance costs. (via http://python.org/)
Languages in
              C
             27%


                   Javascript
                      12%

                                C++
    Python                      3%
     58%
                            (Pyrex/R/Erlang/Go/Shell)
                                    1%
Why Python?
• Hello World: 1
• Hello World: 1
•            :1
• Hello World: 1
•            :1
•         :1
• Hello World: 1
•            :1
•         :1
•         :3
: 13
: 13
import os
from collections import defaultdict

d = defaultdict(int)

for dirpath, dirnames, filenames in os.walk('.'):
    for filename in filenames:
        path = os.path.join(dirpath, filename)
        ext = os.path.splitext(filename)[1]
        d[ext] += len(list(open(path)))

for ext, n_lines in d.items():
    print ext, n_lines
•
• Pythonic
•
    1. svn ci
    2. svn up
    3. restart
• Web
•
•
•
• Battery Included:             200+
• PyPI: 9613 packages currently
• /            /    /     /         /   /
          /...
• easily extensible
Python
Python
Python

Just kidding :-p
Web Server
Web Server
• python -m SimpleHTTPServer
Web Server
• python -m SimpleHTTPServer
web.py                http://webpy.org/

import web

urls = (
    '/(.*)', 'hello'
)
app = web.application(urls, globals())

class hello:
    def GET(self, name):
        if not name:
             name = 'World'
        return 'Hello, ' + name + '!'

if __name__ == "__main__":
    app.run()
Flask       http://flask.pocoo.org/




import flask import Flask
app = Flask(__name__)

@app.route("/<name>")
def hello(name):
    if not name:
        name = 'World'
    return 'Hello, ' + name + '!'

if __name__ == "__main__":
    app.run()
WSGI
   http://www.python.org/dev/peps/pep-0333/
Why so many Python
  web frameworks?
• Because you can write your own
  framework in 3 hours and a total of 60
  lines of Python code.
• http://bitworking.org/news/
  Why_so_many_Python_web_frameworks
doctest
def cube(x):
    """
    >>> cube(10)
    1000
    """
    return x * x

def _test():
    import doctest
    doctest.testmod()

if __name__ == "__main__":
    _test()
nose    http://somethingaboutorange.com/mrl/projects/nose/




from cube import cube

def test_cube():
    result = cube(10)
    assert result == 1000
numpy           http://numpy.scipy.org/




>>> from numpy import *
>>> A = arange(4).reshape(2, 2)
>>> A
array([[0, 1],
       [2, 3]])
>>> dot(A, A.T)
array([[ 1, 3],
       [ 3, 13]])
ipython          http://numpy.scipy.org/




$ ipython -pylab
In [1]: X = frange(0, 10, 0.1)
In [2]: Y = [sin(x) for x in X]
In [3]: plot(X, Y)
ipython          http://numpy.scipy.org/




$ ipython -pylab
In [1]: X = frange(0, 10, 0.1)
In [2]: Y = [sin(x) for x in X]
In [3]: plot(X, Y)
virtualenv            http://virtualenv.openplans.org/




                                python

$ python go-pylons.py --no-site-packages mydevenv
$ cd mydevenv
$ source bin/activate
(mydevenv)$ paster create -t new9 helloworld
Pyrex/Cython

cdef extern from "math.h"
    double sin(double)

cdef double f(double x):
    return sin(x*x)
Pythonic
>>> import this
The Zen of Python, by Tim Peters         http://bit.ly/pyzencn


Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to
break the rules.
                                          
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.               
In the face of ambiguity, refuse the
temptation to guess.
There should be one-- and
preferably only one --obvious way
to do it.
Although that way may not be           Python
obvious at first unless you're Dutch.    
Now is better than never.
Although never is often better than
*right* now.                            
If the implementation is hard to
explain, it's a bad idea.
If the implementation is easy to        
explain, it may be a good idea.
Namespaces are one honking great
idea -- let's do more of those!
Simple is better than
       complex
class HelloWorld
{
        public static void main(String args[])
        {
           System.out.println("Hello World!");
        }
}
Simple is better than
      complex

    print "Hello World!"
Readability counts
Readability counts

•           {}   end
Readability counts

•                  {}   end
•                 (except "@" for
    decorators)
Readability counts

•                    {}   end
•                  (except "@" for
    decorators)

if limit is not None and len(ids)>limit:
    ids = random.sample(ids, limit)
TOOWTDI
• There (should be) Only One Way To Do It.
• vs. Perlish TIMTOWTDI (There Is More
  Than One Way To Do It)
TOOWTDI
 • There (should be) Only One Way To Do It.
 • vs. Perlish TIMTOWTDI (There Is More
    Than One Way To Do It)

a = [1, 2, 3, 4, 5]
b = []
for i in range(len(a)):
    b.append(a[i]*2)
TOOWTDI
 • There (should be) Only One Way To Do It.
 • vs. Perlish TIMTOWTDI (There Is More
    Than One Way To Do It)

a = [1, 2, 3, 4, 5]
b = []
for i in range(len(a)):
    b.append(a[i]*2)
TOOWTDI
 • There (should be) Only One Way To Do It.
 • vs. Perlish TIMTOWTDI (There Is More
    Than One Way To Do It)

a = [1, 2, 3, 4, 5]       b = []
b = []                    for x in a:
for i in range(len(a)):
                              b.append(x*2)
    b.append(a[i]*2)
TOOWTDI
 • There (should be) Only One Way To Do It.
 • vs. Perlish TIMTOWTDI (There Is More
    Than One Way To Do It)

a = [1, 2, 3, 4, 5]       b = []
b = []                    for x in a:
for i in range(len(a)):
                              b.append(x*2)
    b.append(a[i]*2)
TOOWTDI
• There (should be) Only One Way To Do It.
• vs. Perlish TIMTOWTDI (There Is More
  Than One Way To Do It)



     b = [x*2 for x in a]
http://twitter.com/robbinfan/status/9879724095




http://twitter.com/hongqn/status/9883515681
Python                                                            C


http://www.flickr.com/photos/nicksieger/281055485/   http://www.flickr.com/photos/nicksieger/281055530/
Ruby



http://www.flickr.com/photos/nicksieger/280661836/
Java




http://www.flickr.com/photos/nicksieger/280662707/
Python
• svn
• svn
•       (   list)
• svn
•           (   list)
•       +
• svn
•                    (   list)
•       +
•           parser
config.py
MEMCACHED_ADDR = ['localhost:11211']

from local_config import *
config.py
MEMCACHED_ADDR = ['localhost:11211']

from local_config import *




                                       local_config.py
                                   MEMCACHED_ADDR = [
                                       'frodo:11211',
                                       'sam:11211',
                                       'pippin:11211',
                                       'merry:11211',
                                   ]
config.py
MEMCACHED_ADDR = ['localhost:11211']

from local_config import *




                    .py                local_config.py
            exec                   MEMCACHED_ADDR = [
                                       'frodo:11211',
                                       'sam:11211',
                                       'pippin:11211',
                                       'merry:11211',
                                   ]
•
class GroupUI(object):
    def new_topic(self, request):
        if self.group.can_post(request.user):
            return new_topic_ui(self.group)
        else:
            request.response.set_status(403, "Forbidden")
            return error_403_ui(msg="                 ")

    def join(self, request):
        if self.group.can_join(request.user):
        ...

class Group(object):
    def can_post(self, user):
        return self.group.has_member(user)

    def can_join(self, user):
        return not self.group.has_banned(user)
class GroupUI(object):
    @check_permission('post', msg="               ")
    def new_topic(self, request):
        return new_topic_ui(self.group)

    @check_permission('join', msg="          ")
    def join(self, request):
        ...

class Group(object):
    def can_post(self, user):
        return self.group.has_member(user)

    def can_join(self, user):
        return not self.group.has_banned(user)
decorator
def print_before_exec(func):
    def _(*args, **kwargs):
        print "decorated"
        return func(*args, **kwargs)
    return _

@print_before_exec
def double(x):
    print x*2

double(10)
decorator
def print_before_exec(func):
    def _(*args, **kwargs):
        print "decorated"
        return func(*args, **kwargs)
    return _
                                       decorated
@print_before_exec
def double(x):                         20
    print x*2

double(10)
class check_permission(object):
    def __init__(self, action, msg=None):
        self.action = action
        self.msg = msg

   def __call__(self, func):
       def _(ui, req, *args, **kwargs):
           f = getattr(ui.perm_obj, 'can_' + self.action)
           if f(req.user):
                return func(ui, *args, **kwargs)
           raise BadPermission(ui.perm_obj, self.action,
                                self.msg)
       return _
class check_permission(object):
    def __init__(self, action, msg=None):
        self.action = action
        self.msg = msg

    def __call__(self, func):
        def _(ui, req, *args, **kwargs):
            f = getattr(ui.perm_obj, 'can_' + self.action)
            if f(req.user):
                 return func(ui, *args, **kwargs)
            raise BadPermission(ui.perm_obj, self.action,
                                 self.msg)
        return _
class check_permission(object):
    def __init__(self, action, msg=None):
        self.action = action
        self.msg = msg

   def __call__(self, func):
       def _(ui, req, *args, **kwargs):
           f = getattr(ui.perm_obj, 'can_' + self.action)
           if f(req.user):
               return func(ui, *args, **kwargs)
           raise BadPermission(ui.perm_obj, self.action,
                               self.msg)
       return _
class check_permission(object):
    def __init__(self, action, msg=None):
        self.action = action
        self.msg = msg

   def __call__(self, func):
       def _(ui, req, *args, **kwargs):
           f = getattr(ui.perm_obj, 'can_' + self.action)
           if f(req.user):
                return func(ui, *args, **kwargs)
           raise BadPermission(ui.perm_obj, self.action,
                                self.msg)
       return _
class check_permission(object):
    def __init__(self, action, msg=None):
        self.action = action
        self.msg = msg

   def __call__(self, func):
       def _(ui, req, *args, **kwargs):
           f = getattr(ui.perm_obj, 'can_' + self.action)
           if f(req.user):
                return func(ui, *args, **kwargs)
           raise BadPermission(ui.perm_obj, self.action,
                                self.msg)
       return _
class check_permission(object):
    def __init__(self, action, msg=None):
        self.action = action
        self.msg = msg

   def __call__(self, func):
       def _(ui, req, *args, **kwargs):
           f = getattr(ui.perm_obj, 'can_' + self.action)
           if f(req.user):
                return func(ui, *args, **kwargs)
           raise BadPermission(ui.perm_obj, self.action,
                                self.msg)
       return _
class GroupUI(object):
    @check_permission('post', msg="               ")
    def new_topic(self, request):
        return new_topic_ui(self.group)

    @check_permission('join', msg="          ")
    def join(self, request):
        ...

class Group(object):
    def can_post(self, user):
        return self.group.has_member(user)

    def can_join(self, user):
        return not self.group.has_banned(user)
•
def send_notification_mail(email, subject, body):
    msg = MSG_SEND_MAIL + '0' + email + '0' + subject + '0' + body
    mq.put(msg)



def async_worker():
    msg = mq.get()
    msg = msg.split('0')
    cmd = msg[0]
    if cmd == MSG_SEND_MAIL:
        email, subject, body = msg[1:]
        fromaddr = 'no-reply@douban.com'
        email_body = make_email_body(fromaddr, email, subject, body)
        smtp = smtplib.SMTP('mail')
        smtp.sendmail(fromaddr, email, email_body)
    elif cmd == MSG_xxxx:
        ...
    elif cmd == MSG_yyyy:
        ...
@async
def send_notification_mail(email, subject, body):
    fromaddr = 'no-reply@douban.com'
    email_body = make_email_body(fromaddr, email, subject,
body)
    smtp = smtplib.SMTP('mail')
    smtp.sendmail(fromaddr, email, email_body)
def async(func):
    mod = sys.modules[func.__module__]
    fname = 'origin_' + func.__name__
    mod.__dict__[fname] = func
    def _(*a, **kw):
        body = cPickle.dumps((mod.__name__, fname, a, kw))
        mq.put(body)
    return _


def async_worker():
    modname, fname, a, kw = cPickle.loads(mq.get())
    __import__(modname)
    mod = sys.modules[modname]
    mod.__dict__[fname](*a, **kw)
def async(func):
    mod = sys.modules[func.__module__]
    fname = 'origin_' + func.__name__
    mod.__dict__[fname] = func
    def _(*a, **kw):
        body = cPickle.dumps((mod.__name__, fname, a, kw))
        mq.put(body)
    return _


def async_worker():
    modname, fname, a, kw = cPickle.loads(mq.get())
    __import__(modname)
    mod = sys.modules[modname]
    mod.__dict__[fname](*a, **kw)
def async(func):
    mod = sys.modules[func.__module__]
    fname = 'origin_' + func.__name__
    mod.__dict__[fname] = func
    def _(*a, **kw):
        body = cPickle.dumps((mod.__name__, fname, a, kw))
        mq.put(body)
    return _


def async_worker():
    modname, fname, a, kw = cPickle.loads(mq.get())
    __import__(modname)
    mod = sys.modules[modname]
    mod.__dict__[fname](*a, **kw)
def async(func):
    mod = sys.modules[func.__module__]
    fname = 'origin_' + func.__name__
    mod.__dict__[fname] = func
    def _(*a, **kw):
        body = cPickle.dumps((mod.__name__, fname, a, kw))
        mq.put(body)
    return _


def async_worker():
    modname, fname, a, kw = cPickle.loads(mq.get())
    __import__(modname)
    mod = sys.modules[modname]
    mod.__dict__[fname](*a, **kw)
def async(func):
    mod = sys.modules[func.__module__]
    fname = 'origin_' + func.__name__
    mod.__dict__[fname] = func
    def _(*a, **kw):
        body = cPickle.dumps((mod.__name__, fname, a, kw))
        mq.put(body)
    return _


def async_worker():
    modname, fname, a, kw = cPickle.loads(mq.get())
    __import__(modname)
    mod = sys.modules[modname]
    mod.__dict__[fname](*a, **kw)
def async(func):
    mod = sys.modules[func.__module__]
    fname = 'origin_' + func.__name__
    mod.__dict__[fname] = func
    def _(*a, **kw):
        body = cPickle.dumps((mod.__name__, fname, a, kw))
        mq.put(body)
    return _


def async_worker():
    modname, fname, a, kw = cPickle.loads(mq.get())
    __import__(modname)
    mod = sys.modules[modname]
    mod.__dict__[fname](*a, **kw)
def async(func):
    mod = sys.modules[func.__module__]
    fname = 'origin_' + func.__name__
    mod.__dict__[fname] = func
    def _(*a, **kw):
        body = cPickle.dumps((mod.__name__, fname, a, kw))
        mq.put(body)
    return _


def async_worker():
    modname, fname, a, kw = cPickle.loads(mq.get())
    __import__(modname)
    mod = sys.modules[modname]
    mod.__dict__[fname](*a, **kw)
def async(func):
    mod = sys.modules[func.__module__]
    fname = 'origin_' + func.__name__
    mod.__dict__[fname] = func
    def _(*a, **kw):
        body = cPickle.dumps((mod.__name__, fname, a, kw))
        mq.put(body)
    return _


def async_worker():
    modname, fname, a, kw = cPickle.loads(mq.get())
    __import__(modname)
    mod = sys.modules[modname]
    mod.__dict__[fname](*a, **kw)
• cache   (SQL,   , etc)
def get_latest_review_id():
    review_id = mc.get('latest_review_id')
    if review_id is None:
        review_id = exc_sql("select max(id) from review")
        mc.set('latest_review_id', review_id)
    return review_id
@cache('latest_review_id')
def get_latest_review_id():
    return exc_sql("select max(id) from review")
def cache(key):
    def deco(func):
        def _(*args, **kwargs):
            r = mc.get(key)
            if r is None:
                 r = func(*args, **kwargs)
                 mc.set(key, r)
            return r
        return _
    return deco
def cache(key):
    def deco(func):
        def _(*args, **kwargs):
            r = mc.get(key)
            if r is None:
                r = func(*args, **kwargs)
                mc.set(key, r)
            return r
        return _
    return deco
cache key

def get_review(id):
    key = 'review:%s' % id
    review = mc.get(key)
    if review is None: # cache miss
        id, author_id, text = exc_sql("select id, author_id,
text from review where id=%s", id)
        review = Review(id, author_id, text)
        mc.set(key, review)
    return review
cache key
                       decorator


@cache('review:{id}')
def get_review(id):
    id, author_id, text = exc_sql("select id, author_id,
text from review where id=%s", id)
    return Review(id, author_id, text)
def cache(key_pattern, expire=0):
    def deco(f):
        arg_names, varargs, varkw, defaults = inspect.getargspec(f)
        if varargs or varkw:
            raise Exception("not support varargs")
        gen_key = gen_key_factory(key_pattern, arg_names, defaults)

       def _(*a, **kw):
           key = gen_key(*a, **kw)
           r = mc.get(key)
           if r is None:
                r = f(*a, **kw)
                mc.set(key, r, expire)
           return r
       return _
   return deco
def cache(key_pattern, expire=0):
    def deco(f):
        arg_names, varargs, varkw, defaults = inspect.getargspec(f)
        if varargs or varkw:
            raise Exception("not support varargs")
        gen_key = gen_key_factory(key_pattern, arg_names, defaults)

       def _(*a, **kw):
           key = gen_key(*a, **kw)
           r = mc.get(key)
           if r is None:
                r = f(*a, **kw)
                mc.set(key, r, expire)
           return r
       return _
   return deco
def cache(key_pattern, expire=0):
    def deco(f):
        arg_names, varargs, varkw, defaults = inspect.getargspec(f)
        if varargs or varkw:
            raise Exception("not support varargs")
        gen_key = gen_key_factory(key_pattern, arg_names, defaults)

       def _(*a, **kw):
           key = gen_key(*a, **kw)
           r = mc.get(key)
           if r is None:
                r = f(*a, **kw)
                mc.set(key, r, expire)
           return r
       return _
   return deco
inspect.getargspec
>>> import inspect
>>> def f(a, b=1, c=2):
...     pass
...
>>> inspect.getargspec(f)
ArgSpec(args=['a', 'b', 'c'], varargs=None, keywords=None,
defaults=(1, 2))
>>>
>>>
>>> def f(a, b=1, c=2, *args, **kwargs):
...     pass
...
>>> inspect.getargspec(f)
ArgSpec(args=['a', 'b', 'c'], varargs='args', keywords='kwargs',
defaults=(1, 2))
def cache(key_pattern, expire=0):
    def deco(f):
        arg_names, varargs, varkw, defaults = inspect.getargspec(f)
        if varargs or varkw:
            raise Exception("not support varargs")
        gen_key = gen_key_factory(key_pattern, arg_names, defaults)

       def _(*a, **kw):
           key = gen_key(*a, **kw)
           r = mc.get(key)
           if r is None:
                r = f(*a, **kw)
                mc.set(key, r, expire)
           return r
       return _
   return deco
def cache(key_pattern, expire=0):
    def deco(f):
        arg_names, varargs, varkw, defaults = inspect.getargspec(f)
        if varargs or varkw:
            raise Exception("not support varargs")
        gen_key = gen_key_factory(key_pattern, arg_names, defaults)

       def _(*a, **kw):
           key = gen_key(*a, **kw)
           r = mc.get(key)
           if r is None:

                          hint:
                r = f(*a, **kw)
                mc.set(key, r, expire)
           return r        • str.format in python 2.6:
                                '{id}'.format(id=1) => '1'
       return _
   return deco             • dict(zip(['a', 'b', 'c'], [1, 2, 3]))
                                => {'a': 1, 'b': 2, 'c': 3}
def cache(key_pattern, expire=0):
    def deco(f):
        arg_names, varargs, varkw, defaults = inspect.getargspec(f)
        if varargs or varkw:
            raise Exception("not support varargs")
        gen_key = gen_key_factory(key_pattern, arg_names, defaults)

       def _(*a, **kw):
           key = gen_key(*a, **kw)
           r = mc.get(key)
           if r is None:

                          hint:
                r = f(*a, **kw)
                mc.set(key, r, expire)
           return r        • str.format in python 2.6:
                                '{id}'.format(id=1) => '1'
       return _
   return deco             • dict(zip(['a', 'b', 'c'], [1, 2, 3]))
                                => {'a': 1, 'b': 2, 'c': 3}
def cache(key_pattern, expire=0):
    def deco(f):
        arg_names, varargs, varkw, defaults = inspect.getargspec(f)
        if varargs or varkw:
            raise Exception("not support varargs")
        gen_key = gen_key_factory(key_pattern, arg_names, defaults)

       def _(*a, **kw):
           key = gen_key(*a, **kw)
           r = mc.get(key)
           if r is None:
                r = f(*a, **kw)
                mc.set(key, r, expire)
           return r
       return _
   return deco
• feed       feed
  entry_id
class Feed(object):
    def get_entries(self, limit=10):
        ids = exc_sqls("select id from entry where feed_id=
%s order by id desc limit %s", (self.id, limit))
        return [Entry.get(id) for id in ids]

class FeedCollection(object):
    def get_entries(self, limit=10):
        mixed_entries = []
        for feed in self.feeds:
            entries = feed.get_entries(limit=limit)
            mixed_entries += entries
        mixed_entries.sort(key=lambda e: e.id, reverse=True)
        return mixed_entries[:10]
class Feed(object):
    def get_entries(self, limit=10):
        ids = exc_sqls("select id from entry where feed_id=
%s order by id desc limit %s", (self.id, limit))
        return [Entry.get(id) for id in ids]

class FeedCollection(object):
    def get_entries(self, limit=10):
        mixed_entries = []
        for feed in self.feeds:
            entries = feed.get_entries(limit=limit)
            mixed_entries += entries
        mixed_entries.sort(key=lambda e: e.id, reverse=True)
        return mixed_entries[:10]
class Feed(object):
    def get_entries(self, limit=10):
        ids = exc_sqls("select id from entry where feed_id=
%s order by id desc limit %s", (self.id, limit))
        return [Entry.get(id) for id in ids]

class FeedCollection(object):
    def get_entries(self, limit=10):
        mixed_entries = []
        for feed in self.feeds:
            entries = feed.get_entries(limit=limit)
            mixed_entries += entries
        mixed_entries.sort(key=lambda e: e.id, reverse=True)
        return mixed_entries[:10]
class Feed(object):
    def get_entries(self, limit=10):
        ids = exc_sqls("select id from entry where feed_id=
%s order by id desc limit %s", (self.id, limit))
        return [Entry.get(id) for id in ids]

class FeedCollection(object):
                                                              =
    def get_entries(self, limit=10):
                                         len(self.feeds) * limit
        mixed_entries = []
        for feed in self.feeds:
            entries = feed.get_entries(limit=limit)
            mixed_entries += entries
        mixed_entries.sort(key=lambda e: e.id, reverse=True)
        return mixed_entries[:10]
class Feed(object):
    def get_entries(self, limit=10):
        ids = exc_sqls("select id from entry where feed_id=
%s order by id desc limit %s", (self.id, limit))
        return [Entry.get(id) for id in ids]

class FeedCollection(object):
                                                  Entry.get =
    def get_entries(self, limit=10):
                                        len(self.feeds-1) * limit
        mixed_entries = []
        for feed in self.feeds:
            entries = feed.get_entries(limit=limit)
            mixed_entries += entries
        mixed_entries.sort(key=lambda e: e.id, reverse=True)
        return mixed_entries[:10]
iterator and generator
def fib():
    x, y = 1, 1
    while True:
        yield x
        x, y = y, x+y

def odd(seq):
    return (n for n in seq if n%2)

def less_than(seq, upper_limit):
    for number in seq:
        if number >= upper_limit:
            break
        yield number

print sum(odd(less_than(fib(), 4000000)))
itertools
•   count([n]) --> n, n+1, n+2

•   cycle(p) --> p0, p1, ... plast, p0, p1, ...

•   repeat(elem [,n]) --> elem, elem, elem, ...
    endless or up to n times

•   izip(p, q, ...) --> (p[0], q[0]), (p[1],
    q[1]), ...

•   islice(seq, [start,] stop [, step]) -->
    elements from seq[start:stop:step]

•   ... and more ...
class Feed(object):
    def iter_entries(self):
        start_id = sys.maxint
        while True:
            entry_ids = exc_sqls("select id from entry where
feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id))
            if not entry_ids:
                break
            for entry_id in entry_ids:
                yield Entry.get(entry_id)
            start_id = entry_ids[-1]

class FeedCollection(object):
    def iter_entries(self):
        return imerge(*[feed.iter_entries() for feed in self.feeds])

   def get_entries(self, limit=10):
       return list(islice(self.iter_entries(), limit))
class Feed(object):
    def iter_entries(self):
        start_id = sys.maxint
        while True:
            entry_ids = exc_sqls("select id from entry where
feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id))
            if not entry_ids:
                break
            for entry_id in entry_ids:
                yield Entry.get(entry_id)
            start_id = entry_ids[-1]

class FeedCollection(object):
    def iter_entries(self):
        return imerge(*[feed.iter_entries() for feed in self.feeds])

   def get_entries(self, limit=10):
       return list(islice(self.iter_entries(), limit))
class Feed(object):
    def iter_entries(self):
        start_id = sys.maxint
        while True:
            entry_ids = exc_sqls("select id from entry where
feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id))
            if not entry_ids:
                break
            for entry_id in entry_ids:
                yield Entry.get(entry_id)
            start_id = entry_ids[-1]

class FeedCollection(object):
    def iter_entries(self):
        return imerge(*[feed.iter_entries() for feed in self.feeds])

   def get_entries(self, limit=10):
       return list(islice(self.iter_entries(), limit))
class Feed(object):
    def iter_entries(self):
        start_id = sys.maxint
        while True:
            entry_ids = exc_sqls("select id from entry where
feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id))
            if not entry_ids:
                break
            for entry_id in entry_ids:
                yield Entry.get(entry_id)
            start_id = entry_ids[-1]

class FeedCollection(object):
    def iter_entries(self):
        return imerge(*[feed.iter_entries() for feed in self.feeds])

   def get_entries(self, limit=10):
       return list(islice(self.iter_entries(), limit))
class Feed(object):
    def iter_entries(self):
        start_id = sys.maxint
        while True:
            entry_ids = exc_sqls("select id from entry where
feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id))
            if not entry_ids:
                break
            for entry_id in entry_ids:
                yield Entry.get(entry_id)
            start_id = entry_ids[-1]

class FeedCollection(object):
    def iter_entries(self):
        return imerge(*[feed.iter_entries() for feed in self.feeds])

   def get_entries(self, limit=10):
       return list(islice(self.iter_entries(), limit))
class Feed(object):
    def iter_entries(self):
        start_id = sys.maxint
        while True:
            entry_ids = exc_sqls("select id from entry where
feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id))
            if not entry_ids:
                break
            for entry_id in entry_ids:                              =
                yield Entry.get(entry_id)
                                                len(self.feeds) * 5 ~
            start_id = entry_ids[-1]
                                             len(self.feeds)*5 + limit -5
class FeedCollection(object):
    def iter_entries(self):
        return imerge(*[feed.iter_entries() for feed in self.feeds])

    def get_entries(self, limit=10):
        return list(islice(self.iter_entries(), limit))
class Feed(object):
    def iter_entries(self):
        start_id = sys.maxint
        while True:
            entry_ids = exc_sqls("select id from entry where
feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id))
            if not entry_ids:
                break
            for entry_id in entry_ids:
                yield Entry.get(entry_id)
            start_id = entry_ids[-1]                Entry.get =
                                              0 ~ len(self.feeds)-1
class FeedCollection(object):
    def iter_entries(self):
        return imerge(*[feed.iter_entries() for feed in self.feeds])

   def get_entries(self, limit=10):
       return list(islice(self.iter_entries(), limit))
decorator   generator
•
class User(object):
    def __init__(self, id, username, screen_name, sig):
        self.id = id
        self.username = username
        self.screen_name = screen_name
        self.sig = sig

user = User('1002211', 'hongqn', 'hongqn', "
     ")
cPickle vs. marshal
$ python -m timeit -s '
> from user import user
> from cPickle import dumps, loads
> s = dumps(user, 2)' 
> 'loads(s)'
100000 loops, best of 3: 6.6 usec per loop


$ python -m timeit -s '
> from user import user
> from marshal import dumps, loads
> d = (user.id, user.username, user.screen_name, user.sig)
> s = dumps(d, 2)' 'loads(s)'
1000000 loops, best of 3: 0.9 usec per loop
cPickle vs. marshal
$ python -m timeit -s '
> from user import user
> from cPickle import dumps, loads
> s = dumps(user, 2)' 
> 'loads(s)'

               7
100000 loops, best of 3: 6.6 usec per loop


$ python -m timeit -s '
> from user import user
> from marshal import dumps, loads
> d = (user.id, user.username, user.screen_name, user.sig)
> s = dumps(d, 2)' 'loads(s)'
1000000 loops, best of 3: 0.9 usec per loop
cPickle vs. marshal
$ python -m timeit -s '
> from user import user
> from cPickle import dumps, loads
> s = dumps(user, 2)' 
> 'loads(s)'

               7
100000 loops, best of 3: 6.6 usec per loop


$ python -m timeit -s '
> from user import user
> from marshal import dumps, loads
> d = (user.id, user.username, user.screen_name, user.sig)
> s = dumps(d, 2)' 'loads(s)'
1000000 loops, best of 3: 0.9 usec per loop
cPickle vs. marshal
$ python -c timeit
             '
> import cPickle, marshal
> from user import user
> print "pickle:", len(cPickle.dumps(user, 2))
> print "marshal:", len(marshal.dumps((user.id, 
>           user.username, user.screen_name, user.sig), 2))'
pickle: 129
marshal: 74


                43%
cPickle vs. marshal
$ python -c timeit
             '
> import cPickle, marshal
> from user import user
> print "pickle:", len(cPickle.dumps(user, 2))
> print "marshal:", len(marshal.dumps((user.id, 
>           user.username, user.screen_name, user.sig), 2))'
pickle: 129
marshal: 74


                43%
namedtuple
from collections import namedtuple

User = namedtuple('User', 'id username screen_name sig')

user = User('1002211', 'hongqn', 'hongqn', sig="
        ")

user.username
-> 'hongqn'
__metaclass__
class User(tuple):
    __metaclass__ = NamedTupleMetaClass
    __attrs__ = ['id', 'username', 'screen_name', 'sig']

user = User('1002211', 'hongqn', 'hongqn', sig="
        ")

s = marshal.dumps(user.__marshal__())
User.__load_marshal__(marshal.loads(s))
from operator import itemgetter

class NamedTupleMetaClass(type):
    def __new__(mcs, name, bases, dict):
        assert bases == (tuple,)
        for i, a in enumerate(dict['__attrs__']):
            dict[a] = property(itemgetter(i))
        dict['__slots__'] = ()
        dict['__marshal__'] = tuple
        dict['__load_marshal__'] = classmethod(tuple.__new__)
        dict['__getnewargs__'] = lambda self: tuple(self)
        argtxt = repr(tuple(attrs)).replace("'", "")[1:-1]
        template = """def newfunc(cls, %(argtxt)s):
    return tuple.__new__(cls, (%(argtxt)s))""" % locals()
        namespace = {}
        exec template in namespace
        dict['__new__'] = namespace['newfunc']
        return type.__new__(mcs, name, bases, dict)
Warning!
•        request.get_environ(key)
    • e.g.
      request.get_environ('REMOTE_ADDR')
      --> request.remote_addr
descriptor
•            __get__, __set__
    __delete__
    class Descriptor(object):
        def __get__(self, instance, owner):
            return 'descriptor'

    class Owner(object):
        attr = Descriptor()

    owner = Owner()
    owner.attr --> 'descriptor'
descriptor
• classmethod
• staticmethod
• property
     class C(object):
         def get_x(self):
             return self._x
         def set_x(self, x):
             self._x = x
         x = property(get_x, set_x)
class environ_getter(object):
    def __init__(self, key, default=None):
        self.key = key
        self.default = default

    def __get__(self, obj, objtype):
        if obj is None:
            return self
        return obj.get_environ(self.key, self.default)


class HTTPRequest(quixote.http_request.HTTPRequest):
    for key in ['HTTP_REFERER', 'REMOTE_ADDR',
                 'SERVER_NAME', 'REQUEST_URI', 'HTTP_HOST']:
        locals()[key.lower()] = environ_getter(key)
        locals()
    del key
class environ_getter(object):
    def __init__(self, key, default=None):
        self.key = key
        self.default = default

    def __get__(self, obj, objtype):
        if obj is None:
            return self
        return obj.get_environ(self.key, self.default)


class HTTPRequest(quixote.http_request.HTTPRequest):
    for key in ['HTTP_REFERER', 'REMOTE_ADDR',
                'SERVER_NAME', 'REQUEST_URI', 'HTTP_HOST']:
        locals()[key.lower()] = environ_getter(key)
    del key
•   urllib.urlopen   socks
Monkey Patch
import httplib

orig_connect = httplib.HTTPConnection.connect

def _patched_connect(self):
    if HOSTS_BLOCKED.match(self.host):
        return _connect_via_socks_proxy(self)
    else:
        return orig_connect(self)

def _connect_via_socks_proxy(self):
    ...

httplib.HTTPConnection.connect = _patched_connect
Python
Python

• Pythonic!
Python

• Pythonic!
• Avoid gotchas   http://www.ferg.org/projects/python_gotchas.html
Python

• Pythonic!
• Avoid gotchashttp://www.ferg.org/projects/python_gotchas.html


• Unicode / Character Encoding
Python

• Pythonic!
• Avoid gotchas http://www.ferg.org/projects/python_gotchas.html


• Unicode / Character Encoding
• GIL (Global Interpreter Lock)
Python

• Pythonic!
• Avoid gotchas http://www.ferg.org/projects/python_gotchas.html


• Unicode / Character Encoding
• GIL (Global Interpreter Lock)
• Garbage Collection
•         :Vim / Emacs / Ulipad
•           : subversion / mercurial / git
• wiki/          /          : Trac
•           : Bitten
Python
Implementations
Python
    Implementations
• CPython   http://www.python.org/
Python
     Implementations
• CPython http://www.python.org/


• Unlanden-Swallow      http://code.google.com/p/unladen-swallow/
Python
     Implementations
• CPython http://www.python.org/


• Unlanden-Swallow      http://code.google.com/p/unladen-swallow/


• Stackless Python   http://www.stackless.com/
Python
     Implementations
• CPython http://www.python.org/


• Unlanden-Swallow      http://code.google.com/p/unladen-swallow/


• Stackless Python   http://www.stackless.com/


• IronPython http://ironpython.net/
Python
       Implementations
• CPython     http://www.python.org/


• Unlanden-Swallow          http://code.google.com/p/unladen-swallow/


• Stackless Python       http://www.stackless.com/


• IronPython     http://ironpython.net/

• Jython http://www.jython.org/
Python
       Implementations
• CPython     http://www.python.org/


• Unlanden-Swallow          http://code.google.com/p/unladen-swallow/


• Stackless Python       http://www.stackless.com/


• IronPython     http://ironpython.net/

• Jython http://www.jython.org/
• PyPy http://pypy.org/
Q &A

More Related Content

What's hot

What's hot (20)

Learn 90% of Python in 90 Minutes
Learn 90% of Python in 90 MinutesLearn 90% of Python in 90 Minutes
Learn 90% of Python in 90 Minutes
 
Advanced Python, Part 2
Advanced Python, Part 2Advanced Python, Part 2
Advanced Python, Part 2
 
Python Tricks That You Can't Live Without
Python Tricks That You Can't Live WithoutPython Tricks That You Can't Live Without
Python Tricks That You Can't Live Without
 
Matlab and Python: Basic Operations
Matlab and Python: Basic OperationsMatlab and Python: Basic Operations
Matlab and Python: Basic Operations
 
Python高级编程(二)
Python高级编程(二)Python高级编程(二)
Python高级编程(二)
 
Learn python - for beginners - part-2
Learn python - for beginners - part-2Learn python - for beginners - part-2
Learn python - for beginners - part-2
 
Odessapy2013 - Graph databases and Python
Odessapy2013 - Graph databases and PythonOdessapy2013 - Graph databases and Python
Odessapy2013 - Graph databases and Python
 
Python 표준 라이브러리
Python 표준 라이브러리Python 표준 라이브러리
Python 표준 라이브러리
 
Fun never stops. introduction to haskell programming language
Fun never stops. introduction to haskell programming languageFun never stops. introduction to haskell programming language
Fun never stops. introduction to haskell programming language
 
2015 bioinformatics python_strings_wim_vancriekinge
2015 bioinformatics python_strings_wim_vancriekinge2015 bioinformatics python_strings_wim_vancriekinge
2015 bioinformatics python_strings_wim_vancriekinge
 
python beginner talk slide
python beginner talk slidepython beginner talk slide
python beginner talk slide
 
Begin with Python
Begin with PythonBegin with Python
Begin with Python
 
Python Tutorial
Python TutorialPython Tutorial
Python Tutorial
 
java 8 Hands on Workshop
java 8 Hands on Workshopjava 8 Hands on Workshop
java 8 Hands on Workshop
 
Functional Programming & Event Sourcing - a pair made in heaven
Functional Programming & Event Sourcing - a pair made in heavenFunctional Programming & Event Sourcing - a pair made in heaven
Functional Programming & Event Sourcing - a pair made in heaven
 
Commit ускоривший python 2.7.11 на 30% и новое в python 3.5
Commit ускоривший python 2.7.11 на 30% и новое в python 3.5Commit ускоривший python 2.7.11 на 30% и новое в python 3.5
Commit ускоривший python 2.7.11 на 30% и новое в python 3.5
 
Python 101++: Let's Get Down to Business!
Python 101++: Let's Get Down to Business!Python 101++: Let's Get Down to Business!
Python 101++: Let's Get Down to Business!
 
Learn python in 20 minutes
Learn python in 20 minutesLearn python in 20 minutes
Learn python in 20 minutes
 
AmI 2016 - Python basics
AmI 2016 - Python basicsAmI 2016 - Python basics
AmI 2016 - Python basics
 
Python tour
Python tourPython tour
Python tour
 

Viewers also liked

豆瓣技术架构的发展历程 @ QCon Beijing 2009
豆瓣技术架构的发展历程 @ QCon Beijing 2009豆瓣技术架构的发展历程 @ QCon Beijing 2009
豆瓣技术架构的发展历程 @ QCon Beijing 2009
Qiangning Hong
 
服务框架: Thrift & PasteScript
服务框架: Thrift & PasteScript服务框架: Thrift & PasteScript
服务框架: Thrift & PasteScript
Qiangning Hong
 
Python 编程艺术
Python 编程艺术Python 编程艺术
Python 编程艺术
wilhelmshen
 
前端MVC 豆瓣说
前端MVC 豆瓣说前端MVC 豆瓣说
前端MVC 豆瓣说
Ting Lv
 
डॉ. बाबासाहेब आंबेडकर यांचे खुले पत्र
डॉ. बाबासाहेब आंबेडकर यांचे खुले पत्रडॉ. बाबासाहेब आंबेडकर यांचे खुले पत्र
डॉ. बाबासाहेब आंबेडकर यांचे खुले पत्र
samyaksanvad
 
Triggers
TriggersTriggers
Triggers
work
 

Viewers also liked (20)

豆瓣技术架构的发展历程 @ QCon Beijing 2009
豆瓣技术架构的发展历程 @ QCon Beijing 2009豆瓣技术架构的发展历程 @ QCon Beijing 2009
豆瓣技术架构的发展历程 @ QCon Beijing 2009
 
服务框架: Thrift & PasteScript
服务框架: Thrift & PasteScript服务框架: Thrift & PasteScript
服务框架: Thrift & PasteScript
 
New Design of OneRing
New Design of OneRingNew Design of OneRing
New Design of OneRing
 
OneRing @ OSCamp 2010
OneRing @ OSCamp 2010OneRing @ OSCamp 2010
OneRing @ OSCamp 2010
 
DAE
DAEDAE
DAE
 
Python在豆瓣的应用
Python在豆瓣的应用Python在豆瓣的应用
Python在豆瓣的应用
 
Python PPT
Python PPTPython PPT
Python PPT
 
Introduction to Python
Introduction to PythonIntroduction to Python
Introduction to Python
 
Big Data & NoSQL - EFS'11 (Pavlo Baron)
Big Data & NoSQL - EFS'11 (Pavlo Baron)Big Data & NoSQL - EFS'11 (Pavlo Baron)
Big Data & NoSQL - EFS'11 (Pavlo Baron)
 
Python 编程艺术
Python 编程艺术Python 编程艺术
Python 编程艺术
 
Developing With Django
Developing With DjangoDeveloping With Django
Developing With Django
 
前端MVC 豆瓣说
前端MVC 豆瓣说前端MVC 豆瓣说
前端MVC 豆瓣说
 
डॉ. बाबासाहेब आंबेडकर यांचे खुले पत्र
डॉ. बाबासाहेब आंबेडकर यांचे खुले पत्रडॉ. बाबासाहेब आंबेडकर यांचे खुले पत्र
डॉ. बाबासाहेब आंबेडकर यांचे खुले पत्र
 
合久必分,分久必合
合久必分,分久必合合久必分,分久必合
合久必分,分久必合
 
Bai giang hqtcsdl
Bai giang hqtcsdlBai giang hqtcsdl
Bai giang hqtcsdl
 
Triggers
TriggersTriggers
Triggers
 
Sql tutorial
Sql tutorialSql tutorial
Sql tutorial
 
Code
CodeCode
Code
 
Python
PythonPython
Python
 
Advanced Python : Static and Class Methods
Advanced Python : Static and Class Methods Advanced Python : Static and Class Methods
Advanced Python : Static and Class Methods
 

Similar to Python于Web 2.0网站的应用 - QCon Beijing 2010

Paver: the build tool you missed
Paver: the build tool you missedPaver: the build tool you missed
Paver: the build tool you missed
almadcz
 
Python utan-stodhjul-motorsag
Python utan-stodhjul-motorsagPython utan-stodhjul-motorsag
Python utan-stodhjul-motorsag
niklal
 
Refactoring to Macros with Clojure
Refactoring to Macros with ClojureRefactoring to Macros with Clojure
Refactoring to Macros with Clojure
Dmitry Buzdin
 
Python-GTK
Python-GTKPython-GTK
Python-GTK
Yuren Ju
 

Similar to Python于Web 2.0网站的应用 - QCon Beijing 2010 (20)

A Few of My Favorite (Python) Things
A Few of My Favorite (Python) ThingsA Few of My Favorite (Python) Things
A Few of My Favorite (Python) Things
 
Intro to Python
Intro to PythonIntro to Python
Intro to Python
 
Intro
IntroIntro
Intro
 
Paver: the build tool you missed
Paver: the build tool you missedPaver: the build tool you missed
Paver: the build tool you missed
 
Happy Go Programming
Happy Go ProgrammingHappy Go Programming
Happy Go Programming
 
Snakes for Camels
Snakes for CamelsSnakes for Camels
Snakes for Camels
 
Swift for tensorflow
Swift for tensorflowSwift for tensorflow
Swift for tensorflow
 
Functional Programming inside OOP? It’s possible with Python
Functional Programming inside OOP? It’s possible with PythonFunctional Programming inside OOP? It’s possible with Python
Functional Programming inside OOP? It’s possible with Python
 
Python and Oracle : allies for best of data management
Python and Oracle : allies for best of data managementPython and Oracle : allies for best of data management
Python and Oracle : allies for best of data management
 
Python utan-stodhjul-motorsag
Python utan-stodhjul-motorsagPython utan-stodhjul-motorsag
Python utan-stodhjul-motorsag
 
A Recovering Java Developer Learns to Go
A Recovering Java Developer Learns to GoA Recovering Java Developer Learns to Go
A Recovering Java Developer Learns to Go
 
Code with style
Code with styleCode with style
Code with style
 
How Secure Are Docker Containers?
How Secure Are Docker Containers?How Secure Are Docker Containers?
How Secure Are Docker Containers?
 
Python教程 / Python tutorial
Python教程 / Python tutorialPython教程 / Python tutorial
Python教程 / Python tutorial
 
Go Web Development
Go Web DevelopmentGo Web Development
Go Web Development
 
Refactoring to Macros with Clojure
Refactoring to Macros with ClojureRefactoring to Macros with Clojure
Refactoring to Macros with Clojure
 
Python-GTK
Python-GTKPython-GTK
Python-GTK
 
Thinking in Functions: Functional Programming in Python
Thinking in Functions: Functional Programming in PythonThinking in Functions: Functional Programming in Python
Thinking in Functions: Functional Programming in Python
 
Python Homework Sample
Python Homework SamplePython Homework Sample
Python Homework Sample
 
Python 101 language features and functional programming
Python 101 language features and functional programmingPython 101 language features and functional programming
Python 101 language features and functional programming
 

Recently uploaded

CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
Wonjun Hwang
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
FIDO Alliance
 
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
Muhammad Subhan
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
panagenda
 

Recently uploaded (20)

CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
 
الأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهالأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهله
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
“Iamnobody89757” Understanding the Mysterious of Digital Identity.pdf
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptx
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptx
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
2024 May Patch Tuesday
2024 May Patch Tuesday2024 May Patch Tuesday
2024 May Patch Tuesday
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage Intacct
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptx
 
How to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in PakistanHow to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in Pakistan
 
How we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdfHow we scaled to 80K users by doing nothing!.pdf
How we scaled to 80K users by doing nothing!.pdf
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
The Ultimate Prompt Engineering Guide for Generative AI: Get the Most Out of ...
 
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptx
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
 
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
(Explainable) Data-Centric AI: what are you explaininhg, and to whom?
 
Generative AI Use Cases and Applications.pdf
Generative AI Use Cases and Applications.pdfGenerative AI Use Cases and Applications.pdf
Generative AI Use Cases and Applications.pdf
 

Python于Web 2.0网站的应用 - QCon Beijing 2010

  • 1. Python Web 2.0 QCon Beijing 2010 http://www.flickr.com/photos/arnolouise/2986467632/
  • 2. About Me • Python • 2002 Python • 2004 Python • http://www.douban.com/ people/hongqn/ • hongqn@douban.com • http://twitter.com/hongqn
  • 3. Python • Python is a programming language that lets you work more quickly and integrate your systems more effectively. You can learn to use Python and see almost immediate gains in productivity and lower maintenance costs. (via http://python.org/)
  • 4. Languages in C 27% Javascript 12% C++ Python 3% 58% (Pyrex/R/Erlang/Go/Shell) 1%
  • 6.
  • 8. • Hello World: 1 • :1
  • 9. • Hello World: 1 • :1 • :1
  • 10. • Hello World: 1 • :1 • :1 • :3
  • 11.
  • 12. : 13
  • 13. : 13 import os from collections import defaultdict d = defaultdict(int) for dirpath, dirnames, filenames in os.walk('.'): for filename in filenames: path = os.path.join(dirpath, filename) ext = os.path.splitext(filename)[1] d[ext] += len(list(open(path))) for ext, n_lines in d.items(): print ext, n_lines
  • 15. 1. svn ci 2. svn up 3. restart
  • 17. • Battery Included: 200+ • PyPI: 9613 packages currently • / / / / / / /... • easily extensible
  • 21.
  • 23. Web Server • python -m SimpleHTTPServer
  • 24. Web Server • python -m SimpleHTTPServer
  • 25. web.py http://webpy.org/ import web urls = ( '/(.*)', 'hello' ) app = web.application(urls, globals()) class hello: def GET(self, name): if not name: name = 'World' return 'Hello, ' + name + '!' if __name__ == "__main__": app.run()
  • 26. Flask http://flask.pocoo.org/ import flask import Flask app = Flask(__name__) @app.route("/<name>") def hello(name): if not name: name = 'World' return 'Hello, ' + name + '!' if __name__ == "__main__": app.run()
  • 27. WSGI http://www.python.org/dev/peps/pep-0333/
  • 28. Why so many Python web frameworks? • Because you can write your own framework in 3 hours and a total of 60 lines of Python code. • http://bitworking.org/news/ Why_so_many_Python_web_frameworks
  • 29. doctest def cube(x): """ >>> cube(10) 1000 """ return x * x def _test(): import doctest doctest.testmod() if __name__ == "__main__": _test()
  • 30. nose http://somethingaboutorange.com/mrl/projects/nose/ from cube import cube def test_cube(): result = cube(10) assert result == 1000
  • 31. numpy http://numpy.scipy.org/ >>> from numpy import * >>> A = arange(4).reshape(2, 2) >>> A array([[0, 1], [2, 3]]) >>> dot(A, A.T) array([[ 1, 3], [ 3, 13]])
  • 32. ipython http://numpy.scipy.org/ $ ipython -pylab In [1]: X = frange(0, 10, 0.1) In [2]: Y = [sin(x) for x in X] In [3]: plot(X, Y)
  • 33. ipython http://numpy.scipy.org/ $ ipython -pylab In [1]: X = frange(0, 10, 0.1) In [2]: Y = [sin(x) for x in X] In [3]: plot(X, Y)
  • 34. virtualenv http://virtualenv.openplans.org/ python $ python go-pylons.py --no-site-packages mydevenv $ cd mydevenv $ source bin/activate (mydevenv)$ paster create -t new9 helloworld
  • 35. Pyrex/Cython cdef extern from "math.h" double sin(double) cdef double f(double x): return sin(x*x)
  • 37. >>> import this The Zen of Python, by Tim Peters http://bit.ly/pyzencn Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules.   Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced.  
  • 38. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be Python obvious at first unless you're Dutch.   Now is better than never. Although never is often better than *right* now.   If the implementation is hard to explain, it's a bad idea. If the implementation is easy to   explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those!
  • 39. Simple is better than complex class HelloWorld { public static void main(String args[]) { System.out.println("Hello World!"); } }
  • 40. Simple is better than complex print "Hello World!"
  • 43. Readability counts • {} end • (except "@" for decorators)
  • 44. Readability counts • {} end • (except "@" for decorators) if limit is not None and len(ids)>limit: ids = random.sample(ids, limit)
  • 45. TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It)
  • 46. TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It) a = [1, 2, 3, 4, 5] b = [] for i in range(len(a)): b.append(a[i]*2)
  • 47. TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It) a = [1, 2, 3, 4, 5] b = [] for i in range(len(a)): b.append(a[i]*2)
  • 48. TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It) a = [1, 2, 3, 4, 5] b = [] b = [] for x in a: for i in range(len(a)): b.append(x*2) b.append(a[i]*2)
  • 49. TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It) a = [1, 2, 3, 4, 5] b = [] b = [] for x in a: for i in range(len(a)): b.append(x*2) b.append(a[i]*2)
  • 50. TOOWTDI • There (should be) Only One Way To Do It. • vs. Perlish TIMTOWTDI (There Is More Than One Way To Do It) b = [x*2 for x in a]
  • 52. Python C http://www.flickr.com/photos/nicksieger/281055485/ http://www.flickr.com/photos/nicksieger/281055530/
  • 56.
  • 58. • svn • ( list)
  • 59. • svn • ( list) • +
  • 60. • svn • ( list) • + • parser
  • 62. config.py MEMCACHED_ADDR = ['localhost:11211'] from local_config import * local_config.py MEMCACHED_ADDR = [ 'frodo:11211', 'sam:11211', 'pippin:11211', 'merry:11211', ]
  • 63. config.py MEMCACHED_ADDR = ['localhost:11211'] from local_config import * .py local_config.py exec MEMCACHED_ADDR = [ 'frodo:11211', 'sam:11211', 'pippin:11211', 'merry:11211', ]
  • 64.
  • 65. class GroupUI(object): def new_topic(self, request): if self.group.can_post(request.user): return new_topic_ui(self.group) else: request.response.set_status(403, "Forbidden") return error_403_ui(msg=" ") def join(self, request): if self.group.can_join(request.user): ... class Group(object): def can_post(self, user): return self.group.has_member(user) def can_join(self, user): return not self.group.has_banned(user)
  • 66. class GroupUI(object): @check_permission('post', msg=" ") def new_topic(self, request): return new_topic_ui(self.group) @check_permission('join', msg=" ") def join(self, request): ... class Group(object): def can_post(self, user): return self.group.has_member(user) def can_join(self, user): return not self.group.has_banned(user)
  • 67. decorator def print_before_exec(func): def _(*args, **kwargs): print "decorated" return func(*args, **kwargs) return _ @print_before_exec def double(x): print x*2 double(10)
  • 68. decorator def print_before_exec(func): def _(*args, **kwargs): print "decorated" return func(*args, **kwargs) return _ decorated @print_before_exec def double(x): 20 print x*2 double(10)
  • 69. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, req, *args, **kwargs): f = getattr(ui.perm_obj, 'can_' + self.action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
  • 70. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, req, *args, **kwargs): f = getattr(ui.perm_obj, 'can_' + self.action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
  • 71. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, req, *args, **kwargs): f = getattr(ui.perm_obj, 'can_' + self.action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
  • 72. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, req, *args, **kwargs): f = getattr(ui.perm_obj, 'can_' + self.action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
  • 73. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, req, *args, **kwargs): f = getattr(ui.perm_obj, 'can_' + self.action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
  • 74. class check_permission(object): def __init__(self, action, msg=None): self.action = action self.msg = msg def __call__(self, func): def _(ui, req, *args, **kwargs): f = getattr(ui.perm_obj, 'can_' + self.action) if f(req.user): return func(ui, *args, **kwargs) raise BadPermission(ui.perm_obj, self.action, self.msg) return _
  • 75. class GroupUI(object): @check_permission('post', msg=" ") def new_topic(self, request): return new_topic_ui(self.group) @check_permission('join', msg=" ") def join(self, request): ... class Group(object): def can_post(self, user): return self.group.has_member(user) def can_join(self, user): return not self.group.has_banned(user)
  • 76.
  • 77. def send_notification_mail(email, subject, body): msg = MSG_SEND_MAIL + '0' + email + '0' + subject + '0' + body mq.put(msg) def async_worker(): msg = mq.get() msg = msg.split('0') cmd = msg[0] if cmd == MSG_SEND_MAIL: email, subject, body = msg[1:] fromaddr = 'no-reply@douban.com' email_body = make_email_body(fromaddr, email, subject, body) smtp = smtplib.SMTP('mail') smtp.sendmail(fromaddr, email, email_body) elif cmd == MSG_xxxx: ... elif cmd == MSG_yyyy: ...
  • 78. @async def send_notification_mail(email, subject, body): fromaddr = 'no-reply@douban.com' email_body = make_email_body(fromaddr, email, subject, body) smtp = smtplib.SMTP('mail') smtp.sendmail(fromaddr, email, email_body)
  • 79. def async(func): mod = sys.modules[func.__module__] fname = 'origin_' + func.__name__ mod.__dict__[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _ def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod.__dict__[fname](*a, **kw)
  • 80. def async(func): mod = sys.modules[func.__module__] fname = 'origin_' + func.__name__ mod.__dict__[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _ def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod.__dict__[fname](*a, **kw)
  • 81. def async(func): mod = sys.modules[func.__module__] fname = 'origin_' + func.__name__ mod.__dict__[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _ def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod.__dict__[fname](*a, **kw)
  • 82. def async(func): mod = sys.modules[func.__module__] fname = 'origin_' + func.__name__ mod.__dict__[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _ def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod.__dict__[fname](*a, **kw)
  • 83. def async(func): mod = sys.modules[func.__module__] fname = 'origin_' + func.__name__ mod.__dict__[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _ def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod.__dict__[fname](*a, **kw)
  • 84. def async(func): mod = sys.modules[func.__module__] fname = 'origin_' + func.__name__ mod.__dict__[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _ def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod.__dict__[fname](*a, **kw)
  • 85. def async(func): mod = sys.modules[func.__module__] fname = 'origin_' + func.__name__ mod.__dict__[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _ def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod.__dict__[fname](*a, **kw)
  • 86. def async(func): mod = sys.modules[func.__module__] fname = 'origin_' + func.__name__ mod.__dict__[fname] = func def _(*a, **kw): body = cPickle.dumps((mod.__name__, fname, a, kw)) mq.put(body) return _ def async_worker(): modname, fname, a, kw = cPickle.loads(mq.get()) __import__(modname) mod = sys.modules[modname] mod.__dict__[fname](*a, **kw)
  • 87. • cache (SQL, , etc)
  • 88. def get_latest_review_id(): review_id = mc.get('latest_review_id') if review_id is None: review_id = exc_sql("select max(id) from review") mc.set('latest_review_id', review_id) return review_id
  • 89. @cache('latest_review_id') def get_latest_review_id(): return exc_sql("select max(id) from review")
  • 90. def cache(key): def deco(func): def _(*args, **kwargs): r = mc.get(key) if r is None: r = func(*args, **kwargs) mc.set(key, r) return r return _ return deco
  • 91. def cache(key): def deco(func): def _(*args, **kwargs): r = mc.get(key) if r is None: r = func(*args, **kwargs) mc.set(key, r) return r return _ return deco
  • 92. cache key def get_review(id): key = 'review:%s' % id review = mc.get(key) if review is None: # cache miss id, author_id, text = exc_sql("select id, author_id, text from review where id=%s", id) review = Review(id, author_id, text) mc.set(key, review) return review
  • 93. cache key decorator @cache('review:{id}') def get_review(id): id, author_id, text = exc_sql("select id, author_id, text from review where id=%s", id) return Review(id, author_id, text)
  • 94. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: r = f(*a, **kw) mc.set(key, r, expire) return r return _ return deco
  • 95. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: r = f(*a, **kw) mc.set(key, r, expire) return r return _ return deco
  • 96. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: r = f(*a, **kw) mc.set(key, r, expire) return r return _ return deco
  • 97. inspect.getargspec >>> import inspect >>> def f(a, b=1, c=2): ... pass ... >>> inspect.getargspec(f) ArgSpec(args=['a', 'b', 'c'], varargs=None, keywords=None, defaults=(1, 2)) >>> >>> >>> def f(a, b=1, c=2, *args, **kwargs): ... pass ... >>> inspect.getargspec(f) ArgSpec(args=['a', 'b', 'c'], varargs='args', keywords='kwargs', defaults=(1, 2))
  • 98. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: r = f(*a, **kw) mc.set(key, r, expire) return r return _ return deco
  • 99. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: hint: r = f(*a, **kw) mc.set(key, r, expire) return r • str.format in python 2.6: '{id}'.format(id=1) => '1' return _ return deco • dict(zip(['a', 'b', 'c'], [1, 2, 3])) => {'a': 1, 'b': 2, 'c': 3}
  • 100. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: hint: r = f(*a, **kw) mc.set(key, r, expire) return r • str.format in python 2.6: '{id}'.format(id=1) => '1' return _ return deco • dict(zip(['a', 'b', 'c'], [1, 2, 3])) => {'a': 1, 'b': 2, 'c': 3}
  • 101. def cache(key_pattern, expire=0): def deco(f): arg_names, varargs, varkw, defaults = inspect.getargspec(f) if varargs or varkw: raise Exception("not support varargs") gen_key = gen_key_factory(key_pattern, arg_names, defaults) def _(*a, **kw): key = gen_key(*a, **kw) r = mc.get(key) if r is None: r = f(*a, **kw) mc.set(key, r, expire) return r return _ return deco
  • 102. • feed feed entry_id
  • 103. class Feed(object): def get_entries(self, limit=10): ids = exc_sqls("select id from entry where feed_id= %s order by id desc limit %s", (self.id, limit)) return [Entry.get(id) for id in ids] class FeedCollection(object): def get_entries(self, limit=10): mixed_entries = [] for feed in self.feeds: entries = feed.get_entries(limit=limit) mixed_entries += entries mixed_entries.sort(key=lambda e: e.id, reverse=True) return mixed_entries[:10]
  • 104. class Feed(object): def get_entries(self, limit=10): ids = exc_sqls("select id from entry where feed_id= %s order by id desc limit %s", (self.id, limit)) return [Entry.get(id) for id in ids] class FeedCollection(object): def get_entries(self, limit=10): mixed_entries = [] for feed in self.feeds: entries = feed.get_entries(limit=limit) mixed_entries += entries mixed_entries.sort(key=lambda e: e.id, reverse=True) return mixed_entries[:10]
  • 105. class Feed(object): def get_entries(self, limit=10): ids = exc_sqls("select id from entry where feed_id= %s order by id desc limit %s", (self.id, limit)) return [Entry.get(id) for id in ids] class FeedCollection(object): def get_entries(self, limit=10): mixed_entries = [] for feed in self.feeds: entries = feed.get_entries(limit=limit) mixed_entries += entries mixed_entries.sort(key=lambda e: e.id, reverse=True) return mixed_entries[:10]
  • 106. class Feed(object): def get_entries(self, limit=10): ids = exc_sqls("select id from entry where feed_id= %s order by id desc limit %s", (self.id, limit)) return [Entry.get(id) for id in ids] class FeedCollection(object): = def get_entries(self, limit=10): len(self.feeds) * limit mixed_entries = [] for feed in self.feeds: entries = feed.get_entries(limit=limit) mixed_entries += entries mixed_entries.sort(key=lambda e: e.id, reverse=True) return mixed_entries[:10]
  • 107. class Feed(object): def get_entries(self, limit=10): ids = exc_sqls("select id from entry where feed_id= %s order by id desc limit %s", (self.id, limit)) return [Entry.get(id) for id in ids] class FeedCollection(object): Entry.get = def get_entries(self, limit=10): len(self.feeds-1) * limit mixed_entries = [] for feed in self.feeds: entries = feed.get_entries(limit=limit) mixed_entries += entries mixed_entries.sort(key=lambda e: e.id, reverse=True) return mixed_entries[:10]
  • 108. iterator and generator def fib(): x, y = 1, 1 while True: yield x x, y = y, x+y def odd(seq): return (n for n in seq if n%2) def less_than(seq, upper_limit): for number in seq: if number >= upper_limit: break yield number print sum(odd(less_than(fib(), 4000000)))
  • 109. itertools • count([n]) --> n, n+1, n+2 • cycle(p) --> p0, p1, ... plast, p0, p1, ... • repeat(elem [,n]) --> elem, elem, elem, ... endless or up to n times • izip(p, q, ...) --> (p[0], q[0]), (p[1], q[1]), ... • islice(seq, [start,] stop [, step]) --> elements from seq[start:stop:step] • ... and more ...
  • 110. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry where feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1] class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
  • 111. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry where feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1] class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
  • 112. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry where feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1] class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
  • 113. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry where feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1] class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
  • 114. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry where feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1] class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
  • 115. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry where feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: = yield Entry.get(entry_id) len(self.feeds) * 5 ~ start_id = entry_ids[-1] len(self.feeds)*5 + limit -5 class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
  • 116. class Feed(object): def iter_entries(self): start_id = sys.maxint while True: entry_ids = exc_sqls("select id from entry where feed_id=%s and id<%s order by id desc limit 5", (self.id, start_id)) if not entry_ids: break for entry_id in entry_ids: yield Entry.get(entry_id) start_id = entry_ids[-1] Entry.get = 0 ~ len(self.feeds)-1 class FeedCollection(object): def iter_entries(self): return imerge(*[feed.iter_entries() for feed in self.feeds]) def get_entries(self, limit=10): return list(islice(self.iter_entries(), limit))
  • 117. decorator generator
  • 118.
  • 119. class User(object): def __init__(self, id, username, screen_name, sig): self.id = id self.username = username self.screen_name = screen_name self.sig = sig user = User('1002211', 'hongqn', 'hongqn', " ")
  • 120. cPickle vs. marshal $ python -m timeit -s ' > from user import user > from cPickle import dumps, loads > s = dumps(user, 2)' > 'loads(s)' 100000 loops, best of 3: 6.6 usec per loop $ python -m timeit -s ' > from user import user > from marshal import dumps, loads > d = (user.id, user.username, user.screen_name, user.sig) > s = dumps(d, 2)' 'loads(s)' 1000000 loops, best of 3: 0.9 usec per loop
  • 121. cPickle vs. marshal $ python -m timeit -s ' > from user import user > from cPickle import dumps, loads > s = dumps(user, 2)' > 'loads(s)' 7 100000 loops, best of 3: 6.6 usec per loop $ python -m timeit -s ' > from user import user > from marshal import dumps, loads > d = (user.id, user.username, user.screen_name, user.sig) > s = dumps(d, 2)' 'loads(s)' 1000000 loops, best of 3: 0.9 usec per loop
  • 122. cPickle vs. marshal $ python -m timeit -s ' > from user import user > from cPickle import dumps, loads > s = dumps(user, 2)' > 'loads(s)' 7 100000 loops, best of 3: 6.6 usec per loop $ python -m timeit -s ' > from user import user > from marshal import dumps, loads > d = (user.id, user.username, user.screen_name, user.sig) > s = dumps(d, 2)' 'loads(s)' 1000000 loops, best of 3: 0.9 usec per loop
  • 123. cPickle vs. marshal $ python -c timeit ' > import cPickle, marshal > from user import user > print "pickle:", len(cPickle.dumps(user, 2)) > print "marshal:", len(marshal.dumps((user.id, > user.username, user.screen_name, user.sig), 2))' pickle: 129 marshal: 74 43%
  • 124. cPickle vs. marshal $ python -c timeit ' > import cPickle, marshal > from user import user > print "pickle:", len(cPickle.dumps(user, 2)) > print "marshal:", len(marshal.dumps((user.id, > user.username, user.screen_name, user.sig), 2))' pickle: 129 marshal: 74 43%
  • 125. namedtuple from collections import namedtuple User = namedtuple('User', 'id username screen_name sig') user = User('1002211', 'hongqn', 'hongqn', sig=" ") user.username -> 'hongqn'
  • 126. __metaclass__ class User(tuple): __metaclass__ = NamedTupleMetaClass __attrs__ = ['id', 'username', 'screen_name', 'sig'] user = User('1002211', 'hongqn', 'hongqn', sig=" ") s = marshal.dumps(user.__marshal__()) User.__load_marshal__(marshal.loads(s))
  • 127. from operator import itemgetter class NamedTupleMetaClass(type): def __new__(mcs, name, bases, dict): assert bases == (tuple,) for i, a in enumerate(dict['__attrs__']): dict[a] = property(itemgetter(i)) dict['__slots__'] = () dict['__marshal__'] = tuple dict['__load_marshal__'] = classmethod(tuple.__new__) dict['__getnewargs__'] = lambda self: tuple(self) argtxt = repr(tuple(attrs)).replace("'", "")[1:-1] template = """def newfunc(cls, %(argtxt)s): return tuple.__new__(cls, (%(argtxt)s))""" % locals() namespace = {} exec template in namespace dict['__new__'] = namespace['newfunc'] return type.__new__(mcs, name, bases, dict)
  • 129. request.get_environ(key) • e.g. request.get_environ('REMOTE_ADDR') --> request.remote_addr
  • 130. descriptor • __get__, __set__ __delete__ class Descriptor(object): def __get__(self, instance, owner): return 'descriptor' class Owner(object): attr = Descriptor() owner = Owner() owner.attr --> 'descriptor'
  • 131. descriptor • classmethod • staticmethod • property class C(object): def get_x(self): return self._x def set_x(self, x): self._x = x x = property(get_x, set_x)
  • 132. class environ_getter(object): def __init__(self, key, default=None): self.key = key self.default = default def __get__(self, obj, objtype): if obj is None: return self return obj.get_environ(self.key, self.default) class HTTPRequest(quixote.http_request.HTTPRequest): for key in ['HTTP_REFERER', 'REMOTE_ADDR', 'SERVER_NAME', 'REQUEST_URI', 'HTTP_HOST']: locals()[key.lower()] = environ_getter(key) locals() del key
  • 133. class environ_getter(object): def __init__(self, key, default=None): self.key = key self.default = default def __get__(self, obj, objtype): if obj is None: return self return obj.get_environ(self.key, self.default) class HTTPRequest(quixote.http_request.HTTPRequest): for key in ['HTTP_REFERER', 'REMOTE_ADDR', 'SERVER_NAME', 'REQUEST_URI', 'HTTP_HOST']: locals()[key.lower()] = environ_getter(key) del key
  • 134. urllib.urlopen socks
  • 136. import httplib orig_connect = httplib.HTTPConnection.connect def _patched_connect(self): if HOSTS_BLOCKED.match(self.host): return _connect_via_socks_proxy(self) else: return orig_connect(self) def _connect_via_socks_proxy(self): ... httplib.HTTPConnection.connect = _patched_connect
  • 137. Python
  • 139. Python • Pythonic! • Avoid gotchas http://www.ferg.org/projects/python_gotchas.html
  • 140. Python • Pythonic! • Avoid gotchashttp://www.ferg.org/projects/python_gotchas.html • Unicode / Character Encoding
  • 141. Python • Pythonic! • Avoid gotchas http://www.ferg.org/projects/python_gotchas.html • Unicode / Character Encoding • GIL (Global Interpreter Lock)
  • 142. Python • Pythonic! • Avoid gotchas http://www.ferg.org/projects/python_gotchas.html • Unicode / Character Encoding • GIL (Global Interpreter Lock) • Garbage Collection
  • 143. :Vim / Emacs / Ulipad • : subversion / mercurial / git • wiki/ / : Trac • : Bitten
  • 145. Python Implementations • CPython http://www.python.org/
  • 146. Python Implementations • CPython http://www.python.org/ • Unlanden-Swallow http://code.google.com/p/unladen-swallow/
  • 147. Python Implementations • CPython http://www.python.org/ • Unlanden-Swallow http://code.google.com/p/unladen-swallow/ • Stackless Python http://www.stackless.com/
  • 148. Python Implementations • CPython http://www.python.org/ • Unlanden-Swallow http://code.google.com/p/unladen-swallow/ • Stackless Python http://www.stackless.com/ • IronPython http://ironpython.net/
  • 149. Python Implementations • CPython http://www.python.org/ • Unlanden-Swallow http://code.google.com/p/unladen-swallow/ • Stackless Python http://www.stackless.com/ • IronPython http://ironpython.net/ • Jython http://www.jython.org/
  • 150. Python Implementations • CPython http://www.python.org/ • Unlanden-Swallow http://code.google.com/p/unladen-swallow/ • Stackless Python http://www.stackless.com/ • IronPython http://ironpython.net/ • Jython http://www.jython.org/ • PyPy http://pypy.org/
  • 151. Q &A