This document summarizes a presentation about web development for search products at Taobao. It discusses the complexities of developing for platforms with 260 million pageviews and integrating multiple products. Development is done locally using Git while publishing uses Subversion for permission control. Thorough testing is emphasized through linting, integration testing, and regression testing. Assets are managed through preprocessing, timestamps, and CDN handling.

1. Web Developing in Search
文河
Frontend Engineer @Taobao
@yyfrankyy
f2e.us
March 11, 2011
. . . . . .

2. Agenda
.. Introduction
1
.. Development
2
.. Testing
3
.. Publish
4
.. Summaries
5
. . . . . .

3. Products
商品搜索 s.taobao.com
商业搜索 s8.taobao.com
产品搜索 s.taobao.com/product
一淘 www.etao.com
开放搜索 open.s.taobao.com
排行榜 top.taobao.com
垂直市场 ..
地图购 ..
. . . . . .

4. It's Simple
It's all about listing and sorting.
. . . . . .

5. But, It's Not That Easy
. . . . . .

6. But, It's Not That Easy
• 260,000,000 pageviews.
. . . . . .

7. But, It's Not That Easy
• 260,000,000 pageviews.
• 10 requirements per week at least.
• 20 buckets * 2 of (s|s8).taobao.com.
• module integrations of 4+ products.
. . . . . .

8. But, It's Not That Easy
• 260,000,000 pageviews.
• 10 requirements per week at least.
• 20 buckets * 2 of (s|s8).taobao.com.
• module integrations of 4+ products.
• 10 * 20 * 2 * 4 = ?
. . . . . .

9. But, It's Not That Easy
. . . . . .

10. But, It's Not That Easy
Stable • No accident
• Less bugs
• Testable
. . . . . .

11. But, It's Not That Easy
Stable • No accident
• Less bugs
• Testable
Maintainable Forward/Backward compatible
• Easy doing experiments
. . . . . .

12. But, It's Not That Easy
Stable • No accident
• Less bugs
• Testable
Maintainable Forward/Backward compatible
• Easy doing experiments
Performance Fast response for end-users
. . . . . .

13. Environments
Split into three:
. . . . . .

14. Environments
Split into three:
.. Development / Co-Working
1
. . . . . .

15. Environments
Split into three:
.. Development / Co-Working
1
.. Testing
2
. . . . . .

16. Environments
Split into three:
.. Development / Co-Working
1
.. Testing
2
.. Publish
3
. . . . . .

17. Version Control
(Git + Subversion)
. . . . . .

18. Git For Development
• More Lightweight
• More Flexible
. . . . . .

19. Git For Development
• More Lightweight
• More Flexible
• Easy Branching
• master
• prepub
• development
. . . . . .

20. Git For Development
• More Lightweight
• More Flexible
• Easy Branching
• master
• prepub
• development
• That's not what SCM should do!
. . . . . .

21. Subversion For Publish
• Permission Control
• Easy Management
. . . . . .

22. Subversion For Publish
• Permission Control
• Easy Management
• That's what SCM should do!
. . . . . .

23. Directory-based
Timestamp
• search/110303/search.js
. . . . . .

24. Directory-based
Timestamp
• search/110303/search.js
• Fix dependencies conflict on publishing
. . . . . .

25. Directory-based
Timestamp
• search/110303/search.js
• Fix dependencies conflict on publishing
• Easy rollback
. . . . . .

26. Control Everything
HTML Demo Center
• mock everything online
Assets Split them into pieces
• js,css,mhtml,images
. . . . . .

27. Control Everything
HTML Demo Center
• mock everything online
Assets Split them into pieces
• js,css,mhtml,images
Scripts Control automation tools
. . . . . .

28. Quarity
(Whitebox Testing)
• gjslint.vim
• JavaScriptLint.vim
• reviewboard
. . . . . .

29. Quarity
(Whitebox Testing)
• gjslint.vim
• JavaScriptLint.vim
• reviewboard
(Blackbox Testing)
• Daily Test (dev)
• Intergration Testing (QA)
. . . . . .

30. Quarity
(Whitebox Testing)
• gjslint.vim
• JavaScriptLint.vim
• reviewboard
(Blackbox Testing)
• Daily Test (dev)
• Intergration Testing (QA)
• Publish Review (dev)
. . . . . .

31. Quarity
(Whitebox Testing)
• gjslint.vim
• JavaScriptLint.vim
• reviewboard
(Blackbox Testing)
• Daily Test (dev)
• Intergration Testing (QA)
• Publish Review (dev)
• Two Step Regression Testing (QA)
. . . . . .

32. Building System
(Ant + Bash)
. . . . . .

33. Building System
(Ant + Bash)
.. Concat (by config)
1
.. Code Preprocessing
2
• Clean Debug Codes
• Handle DataURI
• Compress
..
3 Handle Timestamp
..
4 Sync with Subversion
. . . . . .

34. Building System
(Ant + Bash)
.. Concat (by config)
1
.. Code Preprocessing
2
• Clean Debug Codes
• Handle DataURI
• Compress
..
3 Handle Timestamp
.. Sync with Subversion
4
(a building system need to be..)
• Easy Run/Use
• Configurable
. . . . . .

35. Assets Handler
(PHP + nginx)
. . . . . .

36. Assets Handler
(PHP + nginx)
• CDN Combo Handler
• Local Assets Combo
. . . . . .

37. Summaries
dev Just code it
test Control everything, test carefully
pub Automatic jobs
. . . . . .

38. What's Next?
dev Connect PHP with assets
test UnitTest coverage
pub Frontend Config/Building server
pub Git hooks for easier building(post-receive)
. . . . . .

39. Thank you!
Q&A
. . . . . .