How to Create a High-Speed Template Engine in Python

  • 14,666 views
Uploaded on

I developed Tenjin which is the fastest template engine in Python world. In this presentation, I show you how to make template engine faster with a lot of benchmarks.

I developed Tenjin which is the fastest template engine in Python world. In this presentation, I show you how to make template engine faster with a lot of benchmarks.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
14,666
On Slideshare
0
From Embeds
0
Number of Embeds
5

Actions

Shares
Downloads
78
Comments
0
Likes
11

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. PyconMini JP 2011How to Create a High-SpeedTemplate Engine in PythonPythonmakoto kuwatahttp://www.kuwata-lab.com/
  • 2. Profile @makotokuwata http://www.kwuata-lab.com/ Ruby/PHP/Python programmer Creator of Erubis (*) Python4PHPer (*) default template engine on Rails 3
  • 3. Python Products Tenjin : very fast temlate engine Kook : task utility like Ant/Rake Benchmarker : a good friend for performance Oktest : new-style testing library
  • 4. Tenjin Very fast One file, 2000 lines Full-featured Python 3 support Google App Engine Release 1.0 coming soon! http://www.kuwta-lab.com/tenjin/
  • 5. Benchmark Tenjin 2660.1 Mako 1426.4 Jinja2 1257.6 Templetor 903.0 Cheetah 562.3 Django 114.2 Genshi 55.7 Kid 34.6 0 600 1200 1800 2400 3000Python 2.5.5, MacOS X 10.6 (x86_64), 2GB pages/secTenjin 1.0.0, Mako 0.2.5, Jinja2 2.2.1, Templetor 0.32,Cheetah 2.2.2, Django 1.1.0, Genshi 0.5.1, Kid 0.9.6
  • 6. Benchmarks forString Concatenation
  • 7. append()
  • 8. Benchmark pages/sec append() 0 200 400 600 800 1000
  • 9. extend()
  • 10. Benchmark pages/sec append() extend() 0 200 400 600 800 1000
  • 11. StringIO
  • 12. Benchmark pages/sec append() extend() StringIO 0 200 400 600 800 1000
  • 13. mmap
  • 14. Benchmark pages/sec append() extend() StringIO mmap 0 200 400 600 800 1000
  • 15. Generator
  • 16. Benchmark pages/sec append() extend() StringIO mmap generator 0 200 400 600 800 1000
  • 17. Slice
  • 18. Benchmark pages/sec append() extend() StringIO mmap generator slice[-1:] slice[99999:] 0 200 400 600 800 1000
  • 19. Bound method
  • 20. Benchmark pages/sec append() extend() StringIO mmap generator slice[-1:] slice[99999:]extend() (bound) 0 200 400 600 800 1000
  • 21. SummaryFast bound method >= slice[] > extend()Slow Generator > append() > mmap > StringIO
  • 22. Try Benchmark Script
  • 23. Step by Step toTune-up Template Code
  • 24. HTML Template
  • 25. Python Code
  • 26. Benchmark pages/sec append (singleline) 0 2000 4000 6000 8000 10000 12000
  • 27. Multiple Line String Eliminates method call
  • 28. Benchmark pages/sec append (singleline) append (multiline) 0 2000 4000 6000 8000 10000 12000
  • 29. From append() to extend() Eliminates method call
  • 30. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) 0 2000 4000 6000 8000 10000 12000
  • 31. Bound Method Eliminates fetch method
  • 32. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) 0 2000 4000 6000 8000 10000 12000
  • 33. str() function Necessary in Python!
  • 34. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str 0 2000 4000 6000 8000 10000 12000
  • 35. Local Variable Local var is faster than global/build-in var
  • 36. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str 0 2000 4000 6000 8000 10000 12000
  • 37. Format (% operator) Delete all str() call by % operator
  • 38. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str append + format 0 2000 4000 6000 8000 10000 12000
  • 39. None => Empty String Converts None to empty string
  • 40. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str append + format extend + to_strextend + _to_str=to_str 0 2000 4000 6000 8000 10000 12000
  • 41. Escape HTML
  • 42. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str append + format extend + to_strextend + _to_str=to_str escape_html + str escape_html + to_str 0 2000 4000 6000 8000 10000 12000
  • 43. C Extension Implemented in C webext: http://pypi.python.org/pypi/Webext/
  • 44. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str append + format extend + to_str extend + _to_str=to_str escape_html + str escape_html + to_strwebext.escape_html, to_str webext.escape_html 0 2000 4000 6000 8000 10000 12000
  • 45. Extreme join() Not escaped Be escaped if index % 2 == 0 if index % 2 == 1 (no need to call escape_html() !)
  • 46. Benchmark Not implemeted yet...
  • 47. SummaryString concatenation is not a bottleneck extend() & join() are enough fastBottleneck is str() and escape_html() join() should call str() internally C Extension (webext) is great
  • 48. Other Topics
  • 49. Google says... ... The major web applications we have surveyed have indicated that they bottleneck primarily on template systems, ... Django? http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
  • 50. Case Study #1 http://www.myweightracker.com/ Switch from Django template to Tenjin M, C, Network, etc... Django ed M, C, Network, etc... Spe ! pp Up A 30%https://groups.google.com/group/kuwata-lab-products/browse_thread/thread/b50877a9c56d64c9/60f77b5c9b9f5238
  • 51. Case Study #2 Ruby on Rails 1.2 Remove helper methods by preprocessing M, C, Network, etc... Helper Methods template engine ed M, C, Network, etc... pp Spe ! A Up 1 00%http://jp.rubyist.net/magazine/?0021-Erubis
  • 52. Components of View Layer Just one of them TemplateImportant for Engine More Importantperformance! for performance! Helper Cache Functions Mechanism
  • 53. Preprocessing in Tenjin Convert Execute Called everytime
  • 54. Preprocessing in Tenjin Call function Convert in this stage Execute Func call removed
  • 55. Python v.s. Others plTenjin (Perl) 12108.0pyTenjin+Webext 4179.7 he st ! phpTenjin (PHP) 2788.0 Pe rl i ion ha mppyTenjin (Python) 2682.9 C rbTenjin (Ruby) 2634.8 0 2500 5000 7500 10000 12500 pages/sec
  • 56. Why Perl is so Fast? No need to call str(val) nor val.toString() Bytecode op for string concatenation
  • 57. C Ext v.s. Pure Script plTenjin Pure Perl MobaSiF C ExtTemplate::Toolkit C ExtpyTenjin+Webext Python + C Ext pyTenjin Pure Python Cheetah C Ext No need to impl engine in C rbTenjin Pure Ruby (except helpers) eruby C Ext 0 2500 5000 7500 10000 12500 pages/sec
  • 58. SummaryView layer components Template engine, Helper functions, and Cache mechanismNo need to implement engine in C(except helper functions)Perl is greatDjango temlate engine sucks
  • 59. Appendix Tenjin: fast & full-featured template engine http://www.kuwata-lab.com/tenjin/ Webext: C extension for escape_html() http://pypi.python.org/pypi/Webext/ Benchmarker: a utility for benchmarking http://pypi.python.org/pypi/Benchmarker/
  • 60. Appendix C Ruby http://www.kuwata-lab.com/presen/rubykaigi2007.pdf http://jp.rubyist.net/magazine/?0022-FasterThanC Java LL http://www.kuwata-lab.com/presen/LL2007LT.pdf http://jp.rubyist.net/magazine/?0024-TemplateSystem http://jp.rubyist.net/magazine/?0024-TemplateSystem2
  • 61. thank you