Your SlideShare is downloading. ×
  • Like
How to Create a High-Speed Template Engine in Python
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

How to Create a High-Speed Template Engine in Python

  • 14,884 views
Published

I developed Tenjin which is the fastest template engine in Python world. In this presentation, I show you how to make template engine faster with a lot of benchmarks.

I developed Tenjin which is the fastest template engine in Python world. In this presentation, I show you how to make template engine faster with a lot of benchmarks.

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
14,884
On SlideShare
0
From Embeds
0
Number of Embeds
5

Actions

Shares
Downloads
79
Comments
0
Likes
11

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. PyconMini JP 2011How to Create a High-SpeedTemplate Engine in PythonPythonmakoto kuwatahttp://www.kuwata-lab.com/
  • 2. Profile @makotokuwata http://www.kwuata-lab.com/ Ruby/PHP/Python programmer Creator of Erubis (*) Python4PHPer (*) default template engine on Rails 3
  • 3. Python Products Tenjin : very fast temlate engine Kook : task utility like Ant/Rake Benchmarker : a good friend for performance Oktest : new-style testing library
  • 4. Tenjin Very fast One file, 2000 lines Full-featured Python 3 support Google App Engine Release 1.0 coming soon! http://www.kuwta-lab.com/tenjin/
  • 5. Benchmark Tenjin 2660.1 Mako 1426.4 Jinja2 1257.6 Templetor 903.0 Cheetah 562.3 Django 114.2 Genshi 55.7 Kid 34.6 0 600 1200 1800 2400 3000Python 2.5.5, MacOS X 10.6 (x86_64), 2GB pages/secTenjin 1.0.0, Mako 0.2.5, Jinja2 2.2.1, Templetor 0.32,Cheetah 2.2.2, Django 1.1.0, Genshi 0.5.1, Kid 0.9.6
  • 6. Benchmarks forString Concatenation
  • 7. append()
  • 8. Benchmark pages/sec append() 0 200 400 600 800 1000
  • 9. extend()
  • 10. Benchmark pages/sec append() extend() 0 200 400 600 800 1000
  • 11. StringIO
  • 12. Benchmark pages/sec append() extend() StringIO 0 200 400 600 800 1000
  • 13. mmap
  • 14. Benchmark pages/sec append() extend() StringIO mmap 0 200 400 600 800 1000
  • 15. Generator
  • 16. Benchmark pages/sec append() extend() StringIO mmap generator 0 200 400 600 800 1000
  • 17. Slice
  • 18. Benchmark pages/sec append() extend() StringIO mmap generator slice[-1:] slice[99999:] 0 200 400 600 800 1000
  • 19. Bound method
  • 20. Benchmark pages/sec append() extend() StringIO mmap generator slice[-1:] slice[99999:]extend() (bound) 0 200 400 600 800 1000
  • 21. SummaryFast bound method >= slice[] > extend()Slow Generator > append() > mmap > StringIO
  • 22. Try Benchmark Script
  • 23. Step by Step toTune-up Template Code
  • 24. HTML Template
  • 25. Python Code
  • 26. Benchmark pages/sec append (singleline) 0 2000 4000 6000 8000 10000 12000
  • 27. Multiple Line String Eliminates method call
  • 28. Benchmark pages/sec append (singleline) append (multiline) 0 2000 4000 6000 8000 10000 12000
  • 29. From append() to extend() Eliminates method call
  • 30. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) 0 2000 4000 6000 8000 10000 12000
  • 31. Bound Method Eliminates fetch method
  • 32. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) 0 2000 4000 6000 8000 10000 12000
  • 33. str() function Necessary in Python!
  • 34. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str 0 2000 4000 6000 8000 10000 12000
  • 35. Local Variable Local var is faster than global/build-in var
  • 36. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str 0 2000 4000 6000 8000 10000 12000
  • 37. Format (% operator) Delete all str() call by % operator
  • 38. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str append + format 0 2000 4000 6000 8000 10000 12000
  • 39. None => Empty String Converts None to empty string
  • 40. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str append + format extend + to_strextend + _to_str=to_str 0 2000 4000 6000 8000 10000 12000
  • 41. Escape HTML
  • 42. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str append + format extend + to_strextend + _to_str=to_str escape_html + str escape_html + to_str 0 2000 4000 6000 8000 10000 12000
  • 43. C Extension Implemented in C webext: http://pypi.python.org/pypi/Webext/
  • 44. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str append + format extend + to_str extend + _to_str=to_str escape_html + str escape_html + to_strwebext.escape_html, to_str webext.escape_html 0 2000 4000 6000 8000 10000 12000
  • 45. Extreme join() Not escaped Be escaped if index % 2 == 0 if index % 2 == 1 (no need to call escape_html() !)
  • 46. Benchmark Not implemeted yet...
  • 47. SummaryString concatenation is not a bottleneck extend() & join() are enough fastBottleneck is str() and escape_html() join() should call str() internally C Extension (webext) is great
  • 48. Other Topics
  • 49. Google says... ... The major web applications we have surveyed have indicated that they bottleneck primarily on template systems, ... Django? http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
  • 50. Case Study #1 http://www.myweightracker.com/ Switch from Django template to Tenjin M, C, Network, etc... Django ed M, C, Network, etc... Spe ! pp Up A 30%https://groups.google.com/group/kuwata-lab-products/browse_thread/thread/b50877a9c56d64c9/60f77b5c9b9f5238
  • 51. Case Study #2 Ruby on Rails 1.2 Remove helper methods by preprocessing M, C, Network, etc... Helper Methods template engine ed M, C, Network, etc... pp Spe ! A Up 1 00%http://jp.rubyist.net/magazine/?0021-Erubis
  • 52. Components of View Layer Just one of them TemplateImportant for Engine More Importantperformance! for performance! Helper Cache Functions Mechanism
  • 53. Preprocessing in Tenjin Convert Execute Called everytime
  • 54. Preprocessing in Tenjin Call function Convert in this stage Execute Func call removed
  • 55. Python v.s. Others plTenjin (Perl) 12108.0pyTenjin+Webext 4179.7 he st ! phpTenjin (PHP) 2788.0 Pe rl i ion ha mppyTenjin (Python) 2682.9 C rbTenjin (Ruby) 2634.8 0 2500 5000 7500 10000 12500 pages/sec
  • 56. Why Perl is so Fast? No need to call str(val) nor val.toString() Bytecode op for string concatenation
  • 57. C Ext v.s. Pure Script plTenjin Pure Perl MobaSiF C ExtTemplate::Toolkit C ExtpyTenjin+Webext Python + C Ext pyTenjin Pure Python Cheetah C Ext No need to impl engine in C rbTenjin Pure Ruby (except helpers) eruby C Ext 0 2500 5000 7500 10000 12500 pages/sec
  • 58. SummaryView layer components Template engine, Helper functions, and Cache mechanismNo need to implement engine in C(except helper functions)Perl is greatDjango temlate engine sucks
  • 59. Appendix Tenjin: fast & full-featured template engine http://www.kuwata-lab.com/tenjin/ Webext: C extension for escape_html() http://pypi.python.org/pypi/Webext/ Benchmarker: a utility for benchmarking http://pypi.python.org/pypi/Benchmarker/
  • 60. Appendix C Ruby http://www.kuwata-lab.com/presen/rubykaigi2007.pdf http://jp.rubyist.net/magazine/?0022-FasterThanC Java LL http://www.kuwata-lab.com/presen/LL2007LT.pdf http://jp.rubyist.net/magazine/?0024-TemplateSystem http://jp.rubyist.net/magazine/?0024-TemplateSystem2
  • 61. thank you