PyconMini JP 2011How to Create a High-SpeedTemplate Engine in PythonPythonmakoto kuwatahttp://www.kuwata-lab.com/
Profile @makotokuwata http://www.kwuata-lab.com/ Ruby/PHP/Python programmer Creator of Erubis (*) Python4PHPer             ...
Python Products Tenjin       : very fast temlate engine Kook         : task utility like Ant/Rake Benchmarker : a good fri...
Tenjin Very fast                       	      	     	  	      	    	  One file, 2000 lines   	                        	 	  ...
Benchmark                   Tenjin                                                   2660.1                   Mako        ...
Benchmarks forString Concatenation
append()
Benchmark                          pages/sec  append()             0   200   400   600       800     1000
extend()
Benchmark                           pages/sec  append()   extend()              0   200   400   600       800     1000
StringIO
Benchmark                           pages/sec  append()   extend()   StringIO              0   200   400   600       800  ...
mmap
Benchmark                           pages/sec  append()   extend()   StringIO    mmap              0   200   400   600    ...
Generator
Benchmark                           pages/sec  append()   extend()   StringIO     mmap  generator              0   200   4...
Slice      	  	               	  	       	    	      	          	  	  	       	  	                       	  	      	      ...
Benchmark                               pages/sec    append()     extend()     StringIO       mmap    generator     slice[...
Bound method
Benchmark                                 pages/sec       append()        extend()       StringIO         mmap      genera...
SummaryFast bound method >= slice[] > extend()Slow Generator > append() > mmap > StringIO
Try Benchmark Script 	         	           	  	      	         	  	    	  	         	                	    	  	    	  	    ...
Step by Step toTune-up Template Code
HTML Template	 	 	 	 	 	 	 	 	 	 	 	 	        	    	     	  	    	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	...
Python Code         	  	         	 	 	 	       	          	      	 	 	                          	 	 	 	 	 	               ...
Benchmark                                                 pages/sec  append (singleline)                        0   2000  ...
Multiple Line String      	                        	            	       	                             	  	              El...
Benchmark                                                 pages/sec  append (singleline)   append (multiline)             ...
From append() to extend()     	                               	 	      	                                 	 	  	 	 	 	 	 	 ...
Benchmark                                                 pages/sec  append (singleline)   append (multiline)   extend (un...
Bound Method 	                    	      	         	                    	      	         	                    	      	    ...
Benchmark                                                 pages/sec  append (singleline)   append (multiline)   extend (un...
str() function     	            	       	          	  	 	       	       	  	 	       	       	                 	      	   ...
Benchmark                                                 pages/sec  append (singleline)   append (multiline)   extend (un...
Local Variable     	  	 	 	 	 	 	 	 	 	  	 	 	 	 	 	 	 	 	      	      	                Local var is faster than          ...
Benchmark                                                 pages/sec  append (singleline)   append (multiline)   extend (un...
Format (% operator)     	                               	                     	                  	                      	 ...
Benchmark                                                 pages/sec  append (singleline)   append (multiline)   extend (un...
None => Empty String     	                     Converts None       	                   to empty string 	 	  	  	         	...
Benchmark                                                   pages/sec    append (singleline)     append (multiline)     ex...
Escape HTML    	       	 	 	             	             	      	 	 	 	 	 	 	 	 	 	 	           	    	 	 	 	 	 	 	 	 	 	 	  ...
Benchmark                                                   pages/sec    append (singleline)     append (multiline)     ex...
C Extension                               Implemented in C     	           	           	          	         	             ...
Benchmark                                                      pages/sec        append (singleline)         append (multil...
Extreme join()             Not escaped                          Be escaped          if index % 2 == 0                   if...
Benchmark  Not implemeted yet...
SummaryString concatenation is not a bottleneck  extend() & join() are enough fastBottleneck is str() and escape_html() jo...
Other Topics
Google says...  ... The major web applications we  have surveyed have indicated that  they bottleneck primarily on  templa...
Case Study #1  http://www.myweightracker.com/  Switch from Django template to Tenjin        M, C, Network, etc...         ...
Case Study #2   Ruby on Rails 1.2   Remove helper methods by preprocessing  M, C, Network, etc...                       He...
Components of View Layer                              Just one of them                   TemplateImportant for       Engin...
Preprocessing in Tenjin    Convert              	         	        	     Execute        Called everytime
Preprocessing in Tenjin              Call function    Convert              in this stage                    	     Execute ...
Python v.s. Others   plTenjin (Perl)                                  12108.0pyTenjin+Webext              4179.7          ...
Why Perl is so Fast? No need to call str(val) nor val.toString() Bytecode op for string concatenation   	      	    	  	  ...
C Ext v.s. Pure Script        plTenjin                                                  Pure Perl       MobaSiF           ...
SummaryView layer components Template engine, Helper functions, and Cache mechanismNo need to implement engine in C(except...
Appendix Tenjin: fast & full-featured template engine  http://www.kuwata-lab.com/tenjin/ Webext: C extension for escape_ht...
Appendix C              Ruby     http://www.kuwata-lab.com/presen/rubykaigi2007.pdf     http://jp.rubyist.net/magazine/?00...
thank you
Upcoming SlideShare
Loading in...5
×

How to Create a High-Speed Template Engine in Python

16,082

Published on

I developed Tenjin which is the fastest template engine in Python world. In this presentation, I show you how to make template engine faster with a lot of benchmarks.

Published in: Technology
0 Comments
11 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
16,082
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
82
Comments
0
Likes
11
Embeds 0
No embeds

No notes for slide

How to Create a High-Speed Template Engine in Python

  1. 1. PyconMini JP 2011How to Create a High-SpeedTemplate Engine in PythonPythonmakoto kuwatahttp://www.kuwata-lab.com/
  2. 2. Profile @makotokuwata http://www.kwuata-lab.com/ Ruby/PHP/Python programmer Creator of Erubis (*) Python4PHPer (*) default template engine on Rails 3
  3. 3. Python Products Tenjin : very fast temlate engine Kook : task utility like Ant/Rake Benchmarker : a good friend for performance Oktest : new-style testing library
  4. 4. Tenjin Very fast One file, 2000 lines Full-featured Python 3 support Google App Engine Release 1.0 coming soon! http://www.kuwta-lab.com/tenjin/
  5. 5. Benchmark Tenjin 2660.1 Mako 1426.4 Jinja2 1257.6 Templetor 903.0 Cheetah 562.3 Django 114.2 Genshi 55.7 Kid 34.6 0 600 1200 1800 2400 3000Python 2.5.5, MacOS X 10.6 (x86_64), 2GB pages/secTenjin 1.0.0, Mako 0.2.5, Jinja2 2.2.1, Templetor 0.32,Cheetah 2.2.2, Django 1.1.0, Genshi 0.5.1, Kid 0.9.6
  6. 6. Benchmarks forString Concatenation
  7. 7. append()
  8. 8. Benchmark pages/sec append() 0 200 400 600 800 1000
  9. 9. extend()
  10. 10. Benchmark pages/sec append() extend() 0 200 400 600 800 1000
  11. 11. StringIO
  12. 12. Benchmark pages/sec append() extend() StringIO 0 200 400 600 800 1000
  13. 13. mmap
  14. 14. Benchmark pages/sec append() extend() StringIO mmap 0 200 400 600 800 1000
  15. 15. Generator
  16. 16. Benchmark pages/sec append() extend() StringIO mmap generator 0 200 400 600 800 1000
  17. 17. Slice
  18. 18. Benchmark pages/sec append() extend() StringIO mmap generator slice[-1:] slice[99999:] 0 200 400 600 800 1000
  19. 19. Bound method
  20. 20. Benchmark pages/sec append() extend() StringIO mmap generator slice[-1:] slice[99999:]extend() (bound) 0 200 400 600 800 1000
  21. 21. SummaryFast bound method >= slice[] > extend()Slow Generator > append() > mmap > StringIO
  22. 22. Try Benchmark Script
  23. 23. Step by Step toTune-up Template Code
  24. 24. HTML Template
  25. 25. Python Code
  26. 26. Benchmark pages/sec append (singleline) 0 2000 4000 6000 8000 10000 12000
  27. 27. Multiple Line String Eliminates method call
  28. 28. Benchmark pages/sec append (singleline) append (multiline) 0 2000 4000 6000 8000 10000 12000
  29. 29. From append() to extend() Eliminates method call
  30. 30. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) 0 2000 4000 6000 8000 10000 12000
  31. 31. Bound Method Eliminates fetch method
  32. 32. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) 0 2000 4000 6000 8000 10000 12000
  33. 33. str() function Necessary in Python!
  34. 34. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str 0 2000 4000 6000 8000 10000 12000
  35. 35. Local Variable Local var is faster than global/build-in var
  36. 36. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str 0 2000 4000 6000 8000 10000 12000
  37. 37. Format (% operator) Delete all str() call by % operator
  38. 38. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str append + format 0 2000 4000 6000 8000 10000 12000
  39. 39. None => Empty String Converts None to empty string
  40. 40. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str append + format extend + to_strextend + _to_str=to_str 0 2000 4000 6000 8000 10000 12000
  41. 41. Escape HTML
  42. 42. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str append + format extend + to_strextend + _to_str=to_str escape_html + str escape_html + to_str 0 2000 4000 6000 8000 10000 12000
  43. 43. C Extension Implemented in C webext: http://pypi.python.org/pypi/Webext/
  44. 44. Benchmark pages/sec append (singleline) append (multiline) extend (unbound) extend (bound) extend + str extend + _str=str append + format extend + to_str extend + _to_str=to_str escape_html + str escape_html + to_strwebext.escape_html, to_str webext.escape_html 0 2000 4000 6000 8000 10000 12000
  45. 45. Extreme join() Not escaped Be escaped if index % 2 == 0 if index % 2 == 1 (no need to call escape_html() !)
  46. 46. Benchmark Not implemeted yet...
  47. 47. SummaryString concatenation is not a bottleneck extend() & join() are enough fastBottleneck is str() and escape_html() join() should call str() internally C Extension (webext) is great
  48. 48. Other Topics
  49. 49. Google says... ... The major web applications we have surveyed have indicated that they bottleneck primarily on template systems, ... Django? http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
  50. 50. Case Study #1 http://www.myweightracker.com/ Switch from Django template to Tenjin M, C, Network, etc... Django ed M, C, Network, etc... Spe ! pp Up A 30%https://groups.google.com/group/kuwata-lab-products/browse_thread/thread/b50877a9c56d64c9/60f77b5c9b9f5238
  51. 51. Case Study #2 Ruby on Rails 1.2 Remove helper methods by preprocessing M, C, Network, etc... Helper Methods template engine ed M, C, Network, etc... pp Spe ! A Up 1 00%http://jp.rubyist.net/magazine/?0021-Erubis
  52. 52. Components of View Layer Just one of them TemplateImportant for Engine More Importantperformance! for performance! Helper Cache Functions Mechanism
  53. 53. Preprocessing in Tenjin Convert Execute Called everytime
  54. 54. Preprocessing in Tenjin Call function Convert in this stage Execute Func call removed
  55. 55. Python v.s. Others plTenjin (Perl) 12108.0pyTenjin+Webext 4179.7 he st ! phpTenjin (PHP) 2788.0 Pe rl i ion ha mppyTenjin (Python) 2682.9 C rbTenjin (Ruby) 2634.8 0 2500 5000 7500 10000 12500 pages/sec
  56. 56. Why Perl is so Fast? No need to call str(val) nor val.toString() Bytecode op for string concatenation
  57. 57. C Ext v.s. Pure Script plTenjin Pure Perl MobaSiF C ExtTemplate::Toolkit C ExtpyTenjin+Webext Python + C Ext pyTenjin Pure Python Cheetah C Ext No need to impl engine in C rbTenjin Pure Ruby (except helpers) eruby C Ext 0 2500 5000 7500 10000 12500 pages/sec
  58. 58. SummaryView layer components Template engine, Helper functions, and Cache mechanismNo need to implement engine in C(except helper functions)Perl is greatDjango temlate engine sucks
  59. 59. Appendix Tenjin: fast & full-featured template engine http://www.kuwata-lab.com/tenjin/ Webext: C extension for escape_html() http://pypi.python.org/pypi/Webext/ Benchmarker: a utility for benchmarking http://pypi.python.org/pypi/Benchmarker/
  60. 60. Appendix C Ruby http://www.kuwata-lab.com/presen/rubykaigi2007.pdf http://jp.rubyist.net/magazine/?0022-FasterThanC Java LL http://www.kuwata-lab.com/presen/LL2007LT.pdf http://jp.rubyist.net/magazine/?0024-TemplateSystem http://jp.rubyist.net/magazine/?0024-TemplateSystem2
  61. 61. thank you
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×