This document discusses Python technologies used at Douban. It describes how Python is used for web development, including frameworks like Quixote and tools like Mako templates. It also discusses using Python for system tasks like caching with cmemcached, profiling code, and interacting with databases. Overall Python allows Douban to build scalable web applications and services.

1. Python 应
2011.12

2. About Me
• / hongqn
• 2002 Python
• 2004 Python
• 2006 现
构师
• http://www.douban.com/
people/hongqn/

3. Python and Douban

8. Python in Douban

9. 协 发环

10. 协 发环
+

11. 协 发环
+ +

12. 协 发环
Face to Face
+
+ +

13. 术团队
动 发 术
产 发
QA

14. 动 发 术
产 发
QA

15. Mako

16. Mako
• A very fast template engine

17. Mako
• A very fast template engine
• Mako IS Python

18. 态 处
static(): 动处 态 URL
<link href="${static('/css/douban.css')}"
rel="stylesheet" type="text/css">

19. 态 处
static(): 动处 态 URL
<link href="${static('/css/douban.css')}"
rel="stylesheet" type="text/css">
发环 http://dev.douban.com/css/douban.css

20. 态 处
static(): 动处 态 URL
<link href="${static('/css/douban.css')}"
rel="stylesheet" type="text/css">
发环 http://dev.douban.com/css/douban.css
线 环 http://img3.douban.com/css/packed_douban9922754105.css

21. 联
...html code....
<script>
${istatic('/js/sns/tribe/manage_members.js')|n}
</script>

22. 预编译
@import

23. 预编译
@import pyScss

24. 动 发 术
产 发
QA

25. APNS Agent

26. APNS Agent
• 应 Apple Push Notification
Service (APNS) 讯

27. APNS Agent
• 应 Apple Push Notification
Service (APNS) 讯
• gevent

28. APNS Agent
• 应 Apple Push Notification
Service (APNS) 讯
• gevent
• bottle

29. APNS Agent
• 应 Apple Push Notification
Service (APNS) 讯
• gevent
• bottle
• APNSWrapper

30. 动 发 术
产 发
QA

31. Quixote

32. Quixote
• Pretty old (we are using version 1.2)

33. Quixote
• Pretty old (we are using version 1.2)
• Simple

34. Quixote
• Pretty old (we are using version 1.2)
• Simple
• Fast

35. Quixote
• Pretty old (we are using version 1.2)
• Simple
• Fast
• Stable

36. Quixote
• Pretty old (we are using version 1.2)
• Simple
• Fast
• Stable
• Traversal based

37. Traversal
http://www.douban.com/group/python/members

38. Traversal
http://www.douban.com/group/python/members
/

39. Traversal
http://www.douban.com/group/python/members
/
import luz.group
group

40. Traversal
http://www.douban.com/group/python/members
/
import luz.group
group
_q_lookup(request, 'python')
python

41. Traversal
http://www.douban.com/group/python/members
/
import luz.group
group
_q_lookup(request, 'python')
python
.members(request)
members

42. Traversal
http://www.douban.com/group/python/members
/
import luz.group
group
_q_lookup(request, 'python')
python
._q_access(request)
.members(request)
members

43. Traversal
http://www.douban.com/group/python/members
/
import luz.group
group
raise DeletedGroupError
_q_lookup(request, 'python')

44. Traversal
http://www.douban.com/group/python/members
/
import luz.group
group raise DeletedGroupError
_q_lookup(request, 'python') _q_exception_handler(request, e)

45. Traversal
http://www.douban.com/group/python/members
/
import luz.group
group raise DeletedGroupError
_q_lookup(request, 'python') _q_exception_handler(request, e)
route based url dispatch
录级 进

46. Decorators
@need_post
@require_login
@check_switch('annotation')
@check_permission('admin')
@jsonize
def preview(self, request):

47. Generators

48. Generators
feed feed feed

49. Generators
feed feed feed
iterator iterator
iterator

50. Generators
feed feed feed
iterator iterator
iterator
imerge
iterator

51. Generators
feed feed feed
iterator iterator
iterator
imerge
iterator
pager
list

52. Meta Programming
class MetaPage(object):
__metaclass__ = MetaObserver
class Vote(Commentable, MetaPage):
kind = K_VOTE
kind_name = 'vote'
assert MetaObserver.get_class_by_name('vote') is Vote

53. OneRing
• Build Desktop
Application Using Web
Technology
• HTML5 + CSS + JS
• WebKit + Qt + Python

54. 动 发 术
产 发
QA

55. Restful MongoDB
Service
• 给 业 储
• django-piston
• sleepy

56. 动 发 术
产 发
QA

57. pylint
• 码 态检查
• SQL inject
检查
• XSS 检查

58. onimaru
• 错误
统
• Based on
django-sentry

59. release manager
• 动 tag
• 认测试结
•发 线报
• django powered

60. 动 发 术
产 发
QA

61. cmy - MySQL Client
• MySQL 户 Python/C API编
• select 语
•
• 访问时 python对
• 时 MySQLdb

62. C++ Intergration
• cos_sim load_data.py
def load():
for user, subject, rating in CmyResult(
"select user_id, subject_id, rating from user_interest"):
cos_sim_io.add_edge(user, subject, rating)
Python 载
C++计

63. 动 发 术
产 发
QA

64. Online Profiler
12164 function calls (12109 primitive calls) in 0.250 CPU seconds
Ordered by: internal time, call count
List reduced from 274 to 60 due to restriction <60>
ncalls tottime percall cumtime percall filename:lineno(function)
1000 0.098 0.000 0.098 0.000 cmemcached.py:22(restore)
61 0.064 0.001 0.070 0.001 cmemcached.py:64(get)
14 0.026 0.002 0.121 0.009 cmemcached.py:68(get_multi)
908 0.010 0.000 0.017 0.000 decorator.py:22(gen_key)
908 0.005 0.000 0.033 0.000 decorator.py:39(_)
67 0.004 0.000 0.199 0.003 debug.py:26(_)
898 0.004 0.000 0.004 0.000 format.py:57(_)
953 0.003 0.000 0.095 0.000 cmemcached.py:70(<genexpr>)
1 0.003 0.003 0.149 0.149 __init__.py:1866(get_loc_period_events)
908 0.002 0.000 0.007 0.000 format.py:10(format)
2 0.002 0.001 0.143 0.071 __init__.py:346(gets)
963 0.002 0.000 0.075 0.000 wrapper.py:138(get)
67 0.001 0.000 0.001 0.000 debug.py:36(format_call)
878 0.001 0.000 0.001 0.000 __init__.py:1240(is_published)
8 0.001 0.000 0.128 0.016 wrapper.py:146(get_multi)
233 0.001 0.000 0.001 0.000 cgi.py:1031(escape)

65. DoubanService
• thrift RPC 务
• PasteScript实现 码 务
• https://github.com/twitter/thrift_client
Python 负载 failover
• nagios munin 监

66. Dpark
• 计
• Modeled after Spark, but using Python
• mesos实现资 调
dpark = DparkContext()
rdd = dpark.textFile(filepath)
def filter(line):
return pattern in line
for line in rdd.filter(filter).collect():
sys.stdout.write(line)

67. pypcap + dpkt
• Warm up
MySQL slaves
• Analyze MySQL
issue with real-
time queries

68. DAE
• Modeled after Google AppEngine, but for
internal use only
• virtualenv to manage dependencies
• gunicorn + gevent
• Best practice built-in
• Developing...

69. 编 语 in

70. 编 语 in

71. 编 语 in

72. 编 语 in

73. 编 语 in

74. 编 语 in

75. 编 语 in

76. 编 语 in

77. Q &A

78. Thanks