How to Create a High-Speed Template Engine in Python

17,106
-1

Published on

I developed Tenjin which is the fastest template engine in Python world. In this presentation, I show you how to make template engine faster with a lot of benchmarks.

Published in: Technology
0 Comments
12 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
17,106
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
82
Comments
0
Likes
12
Embeds 0
No embeds

No notes for slide

How to Create a High-Speed Template Engine in Python

  1. 1. PyconMini JP 2011How to Create a High-SpeedTemplate Engine in PythonPythonmakoto kuwatahttp://www.kuwata-lab.com/
  2. 2. Profile @makotokuwata http://www.kwuata-lab.com/ Ruby/PHP/Python programmer Creator of Erubis (*) Python4PHPer (*) default template engine on Rails 3
  3. 3. Python Products Tenjin : very fast temlate engine Kook : task utility like Ant/Rake Benchmarker : a good friend for performance Oktest : new-style testing library
  4. 4. Tenjin Very fast One file, 2000 lines Full-featured Python 3 support Google App Engine Release 1.0 coming soon! http://www.kuwta-lab.com/tenjin/
  5. 5. Benchmark Tenjin 2660.1 Mako 1426.4 Jinja2 1257.6 Templetor 903.0 Cheetah 562.3 Django 114.2 Genshi 55.7 Kid 34.6 0 600 1200 1800 2400 3000Python 2.5.5, MacOS X 10.6 (x86_64), 2GB pages/secTenjin 1.0.0, Mako 0.2.5, Jinja2 2.2.1, Templetor 0.32,Cheetah 2.2.2, Django 1.1.0, Genshi 0.5.1, Kid 0.9.6
  6. 6. Benchmarks forString Concatenation
  7. 7. append()
  8. 8. Benchmark pages/sec append() 0 200 400 600 800 1000
  9. 9. extend()
  10. 10. Benchmark pages/sec append() extend() 0 200 400 600 800 1000
  11. 11. StringIO
  12. 12. Benchmark pages/sec append() extend() StringIO 0 200 400 600 800 1000
  13. 13. mmap
  14. 14. Benchmark pages/sec append() extend() StringIO mmap 0 200 400 600 800 1000
  15. 15. Generator
  16. 16. Benchmark pages/sec append() extend() StringIO mmap generator 0 200 400 600 800 1000
  17. 17. Slice
  18. 18. Benchmark pages/sec append() extend() StringIO mmap generator slice[-1:] slice[99999:] 0 200 400 600 800 1000
  19. 19. Bound method
  20. 20. Benchmark pages/sec append() extend() StringIO mmap generator slice[-1:] slice[99999:]extend() (bound) 0 200 400 600 800 1000
  21. 21. SummaryFast bound method >= slice[] > extend()Slow Generator > append() > mmap > StringIO
  22. 22. Try Benchmark Script
  23. 23. Step by Step toTune-up Template Code
  24. 24. HTML Template
  25. 25. Python Code
  26. 26. Benchmark pages/sec append (singleline) 0 2000 4000 6000 8000 10000 12000
  27. 27. Multiple Line String Eliminates method call
  28. 28. Benchmark pages/sec append (singleline) append (multiline) 0 2000 4000 6000 8000 10000 12000
  29. 29. From append() to extend() Eliminates method call
  30. 30. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) 0 2000 4000 6000 8000 10000 12000
  31. 31. Bound Method Eliminates fetch method
  32. 32. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) 0 2000 4000 6000 8000 10000 12000
  33. 33. str() function Necessary in Python!
  34. 34. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str 0 2000 4000 6000 8000 10000 12000
  35. 35. Local Variable Local var is faster than global/build-in var
  36. 36. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str 0 2000 4000 6000 8000 10000 12000
  37. 37. Format (% operator) Delete all str() call by % operator
  38. 38. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str append + format 0 2000 4000 6000 8000 10000 12000
  39. 39. None => Empty String Converts None to empty string
  40. 40. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str append + format extend + to_strextend + _to_str=to_str 0 2000 4000 6000 8000 10000 12000
  41. 41. Escape HTML
  42. 42. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str append + format extend + to_strextend + _to_str=to_str escape_html + str escape_html + to_str 0 2000 4000 6000 8000 10000 12000
  43. 43. C Extension Implemented in C webext: http://pypi.python.org/pypi/Webext/
  44. 44. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str append + format extend + to_str extend + _to_str=to_str escape_html + str escape_html + to_strwebext.escape_html, to_str webext.escape_html 0 2000 4000 6000 8000 10000 12000
  45. 45. Extreme join() Not escaped Be escaped if index % 2 == 0 if index % 2 == 1 (no need to call escape_html() !)
  46. 46. Benchmark Not implemeted yet...
  47. 47. SummaryString concatenation is not a bottleneck extend() & join() are enough fastBottleneck is str() and escape_html() join() should call str() internally C Extension (webext) is great
  48. 48. Other Topics
  49. 49. Google says... ... The major web applications we have surveyed have indicated that they bottleneck primarily on template systems, ... Django? http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
  50. 50. Case Study #1 http://www.myweightracker.com/ Switch from Django template to Tenjin M, C, Network, etc... Django ed M, C, Network, etc... Spe ! pp Up A 30%https://groups.google.com/group/kuwata-lab-products/browse_thread/thread/b50877a9c56d64c9/60f77b5c9b9f5238
  51. 51. Case Study #2 Ruby on Rails 1.2 Remove helper methods by preprocessing M, C, Network, etc... Helper Methods template engine ed M, C, Network, etc... pp Spe ! A Up 1 00%http://jp.rubyist.net/magazine/?0021-Erubis
  52. 52. Components of View Layer Just one of them TemplateImportant for Engine More Importantperformance! for performance! Helper Cache Functions Mechanism
  53. 53. Preprocessing in Tenjin Convert Execute Called everytime
  54. 54. Preprocessing in Tenjin Call function Convert in this stage Execute Func call removed
  55. 55. Python v.s. Others plTenjin (Perl) 12108.0pyTenjin+Webext 4179.7 he st ! phpTenjin (PHP) 2788.0 Pe rl i ion ha mppyTenjin (Python) 2682.9 C rbTenjin (Ruby) 2634.8 0 2500 5000 7500 10000 12500 pages/sec
  56. 56. Why Perl is so Fast? No need to call str(val) nor val.toString() Bytecode op for string concatenation
  57. 57. C Ext v.s. Pure Script plTenjin Pure Perl MobaSiF C ExtTemplate::Toolkit C ExtpyTenjin+Webext Python + C Ext pyTenjin Pure Python Cheetah C Ext No need to impl engine in C rbTenjin Pure Ruby (except helpers) eruby C Ext 0 2500 5000 7500 10000 12500 pages/sec
  58. 58. SummaryView layer components Template engine, Helper functions, and Cache mechanismNo need to implement engine in C(except helper functions)Perl is greatDjango temlate engine sucks
  59. 59. Appendix Tenjin: fast & full-featured template engine http://www.kuwata-lab.com/tenjin/ Webext: C extension for escape_html() http://pypi.python.org/pypi/Webext/ Benchmarker: a utility for benchmarking http://pypi.python.org/pypi/Benchmarker/
  60. 60. Appendix C Ruby http://www.kuwata-lab.com/presen/rubykaigi2007.pdf http://jp.rubyist.net/magazine/?0022-FasterThanC Java LL http://www.kuwata-lab.com/presen/LL2007LT.pdf http://jp.rubyist.net/magazine/?0024-TemplateSystem http://jp.rubyist.net/magazine/?0024-TemplateSystem2
  61. 61. thank you
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×