How Xslate Works

6,912 views
6,755 views

Published on

My presentation "How Xslate Works" at YAPC::Asia Tokyo 2010

Published in: Technology, Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,912
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
18
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

How Xslate Works

  1. 1. How Xslate works The next generation's template engine YAPC::Asia Tokyo 2010, Day 2, 2010/10/16 10:10-10:50 Fuji, Goro (gfx) [email_address] http://github.com/gfx /
  2. 2. Who am I? <ul><li>Fuji, Goro ( 藤 吾郎 ) </li><ul><li>id:gfx (hatena)
  3. 3. @__gfx__ (twitter) </li></ul><li>I love... </li><ul><li>Perl Internals
  4. 4. Perl/XS
  5. 5. Moose/Mouse (as Any::Moose) </li></ul></ul>
  6. 6. Agenda <ul><li>What is Xslate
  7. 7. How Xslate works </li><ul><li>Execution process
  8. 8. Why so fast </li></ul><li>Futures (TODOs)
  9. 9. Information </li></ul>
  10. 10. What is Xslate? <ul><li>A template engine for Perl5
  11. 11. Written in XS </li><ul><li>But also available in pure Perl (thanks to @maka2_donzoko) </li></ul><li>Fast, safe, easy to enhance
  12. 12. Multi-syntaxes (esp. TT-like syntax)
  13. 13. Kind error messages </li></ul>
  14. 14. What are template engines? <ul><li>Text processor vital for web applicatins
  15. 15. my $var = sprintf 'Hello, %s world', $foo; # <- this
  16. 16. $template = 'Hello, %foo% world?'; # <- this </li><ul><li>$templat =~ s/ %(w+)% / $vars->{$1 }/xmsge; </li></ul></ul>
  17. 17. So why to use Xslate? <ul><li>There are too many template engines on CPAN </li><ul><li>Template::Toolkit 2 (and ::Tiny)
  18. 18. HTML::Template (and ::Pro, ::Compiled, ::JIT)
  19. 19. Text::MicroTemplate (and ::Extended)
  20. 20. Mojo::Template
  21. 21. Text::ClearSilver (a perl binding to clearsilver)
  22. 22. HTML::Mason, Text::MicroMason
  23. 23. Tenjin, Template::Alloy, Template::Sandbox, etc... </li></ul><li>There is no perfect module, though </li></ul>
  24. 24. Pick up the well-known <ul><li>Template::Toolkit 2 </li><ul><li>+: Rich functionality, easy to use, easy to enhance
  25. 25. -: Very slow, too large spec, vulnerable to XSS </li></ul><li>HTML::Template::Pro </li><ul><li>+: Fast, ristricted
  26. 26. -: Hard to use, poor functionality, vulnerable to XSS </li></ul><li>Text::MocroTemplate </li><ul><li>+: Fast enough, easy to install, tolerant to XSS
  27. 27. -: Too free, only usaful to Perl programmers </li></ul></ul>
  28. 28. Comparison with Xslate <ul><li>v.s. Template::Toolkit 2 </li><ul><li>Much faster (100+ times faster)
  29. 29. Tolerant to XSS </li></ul><li>v.s. HTML::Template::Pro </li><ul><li>Even faster
  30. 30. Allows complex expressions, and tolerant to XSS </li></ul><li>v.s. Text::MocroTemplate </li><ul><li>Slightly faster
  31. 31. Ristricted (you cannot call system() nor fork()) </li></ul></ul>
  32. 32. Text::ClearSilver <ul><li>A Perl binding to clearsilver (C library)
  33. 33. ClearSilver is a good template engine,
  34. 34. but the official ClearSilver.pm sucks! </li><ul><li>-> Text::ClearSilver (requested by id:craftworks) </li></ul><li>however, ClearSilver is not perfect </li></ul>
  35. 35. Thus, use Text::Xslate! <ul><li>There is no the best, but is a better, Xslate
  36. 36. Xslate is at least the fastest, the most safe in the template engines on CPAN </li></ul>
  37. 37. Tutorial <ul><li>Basic usage: Text::Xslate->new()->render()
  38. 38. new() accepts: syntax (template syntax) , module (function-based modules), function (additional functions), path (include path), cache (caching level), cache_dir (used for caches)
  39. 39. render() returns a rendered text; cannot output to filehandles directly, nor set calbacks (unlike TT)
  40. 40. See also Text::Xslate and Text::Xslate::Manual </li></ul>
  41. 41. Simplest example #!perl -w use 5.10.0; use strict; use Text::Xslate; my $tx = Text::Xslate-> new (); my %vars = ( lang => 'Xslate', ); # or $tx-> render ($file, vars); say $tx-> render_string (<<'EOT', vars); Hello, <: $lang :> world! EOT # -> Hello, Xslate world!
  42. 42. Xslate templates <ul> :# This is a template with the default syntax Kolon :# an one-line code starts from a colon &quot;:&quot; :# comments start from a hash symbol &quot;#&quot; :# an in-line code is surrounded by ”<:” and &quot;:>&quot; : for $list -> $item { <li> <: $item.name :> = <: $item.value :> </li> : } </ul> : include 'foo.tx' { bar => 'overrided' } :# See also ` perldoc Text::Xslate::Syntax::Kolon `
  43. 43. Demo <ul><li>cpanm Text::Xslate </li><ul><li>requires Any::Moose (Mouse), Data::MessagePack, etc. </li></ul><li>xslate -e 'Hello, <: $ARGV[0] :>' Xslate
  44. 44. xslate -s TTerse -e '[% ARGV.0 %]' 'Hi, TTerse'
  45. 45. See also example files in Text-Xslate/example/ </li></ul>
  46. 46. Details <ul><li>Performance
  47. 47. Safe (XSS tolerance and ristriction)
  48. 48. Multi-syntaxes </li><ul><li>Kolon
  49. 49. TTerse
  50. 50. Clevery </li></ul><li>Enhancement
  51. 51. Template cascading </li></ul>
  52. 52. Performance (1) Run Text-Xslate/benchmark/x-rich-env.pl!
  53. 53. Performance (2) <ul><li>Sam Graham's report ”Template roundup” </li><ul><li>http://www.illusori.co.uk/projects/Template-Roundup/
  54. 54. based on benchmarks with Template::Benchmark </li></ul><li>Accoding to this report, Xslate is fastest on the 'instance_reuse' condition
  55. 55. i.e. Persistent PSGI applications will take the best performance </li></ul>
  56. 56. Safe (1) <ul><li>Risks caused by Template Engines </li><ul><li>XSS: Cross Site Scripting
  57. 57. a type of computer security vulnerability typically found in web applications that enables malicious attackers to inject client-side script into web pages viewed by other users. (by Wikipedia) </li></ul></ul>
  58. 58. Safe (2) <ul><li>Typically caused by missing HTML escaping </li><ul><li>e.g. [% foo | html %] (TT2) </li></ul><li>Problem: TT2 requires explicit escaping </li><ul><li>Explicity sucks! </li></ul></ul>
  59. 59. Safe (3) <ul><li>Solution: Template engines must apply HTML-escaping automatically
  60. 60. -> Text::MicroTemplate, Text::Xslate
  61. 61. This is XSS tolerance
  62. 62. Modern template engines (not only in Perl) should have XSS tolerance, and in fact do so. </li></ul>
  63. 63. How to write safe templates <ul><li>Don't use the 'raw' filter in templates </li><ul><li>Don't do that: <: $foo | raw :>
  64. 64. Escape $foo in perl code, because whether $foo is escaped or not is obscure from templates
  65. 65. The 'raw' filter is provided only for string literals </li></ul></ul>
  66. 66. Multi-syntaxes <ul><li>There are multiple template syntaxes </li><ul><li>Kolon – the default, fully-featured
  67. 67. TTerse – TT2 compatible
  68. 68. Clevery – Smarty (in PHP) compatible </li></ul></ul>
  69. 69. Kolon <ul><li>The default template syntax
  70. 70. All the features are supported
  71. 71. Most optimized </li></ul>
  72. 72. TTerse <ul><li>TT2 compatible (urged by tokuhirom++)
  73. 73. Available by default
  74. 74. More ristricted than the original TT2
  75. 75. No plugins, exceptions, nor mysterious features
  76. 76. As good as Kolon to run-time performance
  77. 77. Highly compatible, but diffrent to the basics of escaping mechanism </li></ul>
  78. 78. Clevery <ul><li>Smarty compatible template syntax
  79. 79. Need to install Text::Clevery
  80. 80. With some overhead </li></ul>Clevery Clevery
  81. 81. Enhancement <ul><li>Add TT2-like methods to Xslate </li><ul><li>Text::Xslate::Bridge::TT2Like </li></ul><li>Function-based modules </li><ul><li>Available via the &quot;module&quot; option
  82. 82. No plugin namespaces are required </li></ul></ul>
  83. 83. Template cascading <ul><li>Inspired by template inheritance of T::MicroTemplate::Extended (typester++) </li><ul><li>Originated from Django, a framework for Python </li></ul><li>More powerful than the 'include' command
  84. 84. Only available in Kolon
  85. 85. See also Text-Xslate/example/{cascade.pl,cascade,tx,base.tx} </li></ul>
  86. 86. How Xslate works <ul><li>Execution process </li><ul><li>Preprocessing
  87. 87. Parsing
  88. 88. Compiling </li><ul><li>Saveing/loading bytecode </li></ul><li>Executing </li></ul><li>Why so fast </li></ul>
  89. 89. Execution Process <: $foo :> <: $foo :> print $foo; <: $foo :> fetch_s &quot;foo&quot; print end print $foo Virtual Machine { foo => 'bar' } ”bar&quot;
  90. 90. Preprocessing <ul><li>&quot; Hello, <: $lang :> world! &quot; is difficult to parse :(
  91. 91. So convert to that first:
  92. 92. &quot; print_raw 'Hello, '; print $lang; print_raw ' world!' &quot;
  93. 93. See &T::X::Parser::preprocess </li></ul>
  94. 94. Parsing <ul><li>Parse &quot; print_raw 'Hello, '; print $lang; print_raw ' world!' &quot; and build syntax tree
  95. 95. Using Top Down Operator Precendence method
  96. 96. Explained in ” Beautiful Code ” (O'REILLY)
  97. 97. &quot; It is easy to use. It feels a lot like Recursive Descent, but with the need for less code and with significantly better performance. &quot;, Douglas Crockford introduced there.
  98. 98. See also http://javascript.crockford.com/tdop/ </li></ul>
  99. 99. Top Down Operator Precedence <ul><li>A top down parsering method
  100. 100. Templates -> Tokens (string) -> Symbols (T::X::Symbol) -> Nodes (T::X::Symbol)
  101. 101. Symbols (e.g. '+' for infix:<+>) know what they do
  102. 102. One easily can extend parsers by adding symbols
  103. 103. The parser entry point is &T::X::Parser::statements </li></ul>
  104. 104. The heart of TDOP <ul><li>A symbol might have denotations: </li><ul><li>std : statement denotation (for statements)
  105. 105. nud : null denotation (for objects and prefixes)
  106. 106. led : left denotation (for infixes and postfixes) </li></ul><li>A &quot;denotation&quot; is a method called by statements() </li></ul>
  107. 107. Statement Denotation <ul><li>e.g. 'if', 'for', 'include' </li></ul>sub std_while { my($parser, $symbol) = @_; # $symbol represents 'while' my $proc = $symbol->clone( arity => 'while' # node type ); $proc->first( # set first child $parser->expression(0) ); $parser->pointy($proc); # parse return $proc; }
  108. 108. Null Denotation <ul><li>e.g. variables, literals, parens </li></ul># '(' expr ')' sub nud_paren { my($parser, $symbol) = @_; # $symbol is '(' my $expr = $parser->expression(0); # $symbol->conterpart is ')' $parser->advance( $symbol->counterpart ); return $expr; }
  109. 109. Left Denotation <ul><li>e.g. 'if', 'for', 'include' </li></ul>sub led_infix { my($parser, $symbol, $left) = @_; return $parser->binary( $symbol, $left, $parser->expression( $symbol->lbp) ); }
  110. 110. Compiling <ul><li>Convert the abstract syntax tree into bytecode
  111. 111. Bytecode is a serialized sequence of opcodes </li><ul><li># Hello, <: $lang :> world!
  112. 112. print_raw_s &quot;Hello, &quot;
  113. 113. fetch_s &quot;lang&quot;
  114. 114. print
  115. 115. print_raw_s &quot; world!&quot;
  116. 116. end </li></ul><li>Also perform optimization </li></ul>
  117. 117. Saveing/Loading Bytecode <ul><li>Precompiling is vital for template engines </li><ul><li>e.g. ClearSilver compiles templates, but it cannot save compiled ones (maybe because it targets CGI) </li></ul><li>Using Data::MessagePack for serialization </li><ul><li>MessagePack: spec for binary data serialization
  118. 118. Like JSON, but fast and small
  119. 119. See also http://msgpack.org/ </li></ul></ul>
  120. 120. Assembling <ul><li>Convert the name of opcode to the address of it
  121. 121. Using the direct threaded code paradigm </li></ul>
  122. 122. Executing <ul><li>Execute by the virtual machine
  123. 123. An opcode (e.g. 'fetch_s') is a block of code with an argument (a sv, int, or address of opcode)
  124. 124. e.g. <: $foo :> ($a is a register, $out is the reuslt) </li><ul><li>fetch_s &quot;foo&quot; # $a = $vars->{foo}
  125. 125. print # $out .= $a </li></ul></ul>
  126. 126. Execution Process <: $foo :> <: $foo :> print $foo; <: $foo :> fetch_s &quot;foo&quot; print end print $foo Virtual Machine { foo => 'bar' } ” bar&quot;
  127. 127. Why so fast <ul><li>Precompiling & optimizing
  128. 128. Virtual Machine paradigm </li><ul><li>Direct threaded code (but gcc only) </li></ul><li>HTML escaping
  129. 129. Preallocation of the output buffer </li></ul>
  130. 130. Optimizing bytecode <ul><li>Constant folding </li><ul><li><: 1 + 2 :> (->compile->) literal_i 3
  131. 131. <: if(0) { ... } :> (->compile->) noop </li></ul><li>Injection of cascading templates </li><ul><li>Cascading templates are statically concatinated
  132. 132. Thus, you cannot pass variables to 'cascade' </li></ul><li>$ ack -i 'optimize' lib/Text/Xslate/Compiler.pm </li></ul>
  133. 133. Virtual Machine <ul><li>Virtual machine is fast!
  134. 134. Register machine (two registers + one stack) </li></ul>
  135. 135. Direct Threaded Code <ul><li>Using direct pointers of labels, not pointers of functions nor indexes of function table </li></ul>
  136. 136. HTML Escaping <ul><li>Important for all the template engines
  137. 137. Using highly optimized routines (kazuho++) </li><ul><li>As fast as memcpy(3)
  138. 138. tx_sv_cat_with_html_escape_force() at Text-Xslate/xs/Text-Xslate.xs </li></ul><li>Experimental hacks for Text::MicroTemplate: </li><ul><li>Rewriteing HTML escaping routines makes TMT faster than HTML::Template::Pro </li></ul></ul>
  139. 139. Preallocation <ul><li>Assumption: a template produces a string, and the size of the string is always the same
  140. 140. Then
  141. 141. First: Save the size of a template
  142. 142. Later: Preallocate the buffer with the saved size </li></ul>
  143. 143. Futures <ul><li>Loop controls
  144. 144. Context controls
  145. 145. Augoment block modifiers
  146. 146. More template syntaxes
  147. 147. Xslate in WAF </li></ul>
  148. 148. Loop controls <ul><li>last, next (redo?)
  149. 149. <: last if $a > 10 :> in Kolon
  150. 150. [% LAST IF a > 10 %] in TTerse </li></ul>
  151. 151. Context controls <ul><li>Xslate calls functions/methods in scalar context, but there are other cases...
  152. 152. postfix:<context>? </li><ul><li><: [ $o->foo() list ] :> </li></ul><li>postfix:<@context>? </li><ul><li><: [ $o->foo() @list ] :> </li></ul><li>Any ideas? </li></ul>
  153. 153. Augment Block Modifiers <ul><li>Xslate already supports 'around', 'before', 'after'
  154. 154. 'augment' like block modifiers are requested </li></ul>
  155. 155. More templat syntaxes <ul><li>HTML::Template (if someone wants)
  156. 156. Django (in Python)
  157. 157. Jinja (in Python)
  158. 158. Any ideas? </li></ul>
  159. 159. Xslate in WAF <ul><li>There are some bridges </li><ul><li>Catalyst::View::Xslate for Catalyst
  160. 160. MojoX::Renderer::Xslate for Mojo
  161. 161. Dancer::Template::Xslate for Dancer
  162. 162. Amon uses Xslate via Tiffany </li></ul><li>More examples are required! </li><ul><li>Help me, hackers! </li></ul></ul>
  163. 163. Information <ul><li>http://xslate.org
  164. 164. Web+DB Press Vol.59 </li></ul>
  165. 165. Vote me! Mac Book Proほしいけどお金がありません!(>_<) もしよかったら投票お願いします! http://yapcasia.org/2010/vote
  166. 166. Questions <ul><li>Q. Perlコードに落とし込むという方法もあるが、なぜわざわざVMを?
  167. 167. A. テンプレートのレンダリングに特化しているので速い。特にエスケープしながらの連結が効果的
  168. 168. Q. XslateのVMの上でPerl6は動かせる?
  169. 169. A. 普通の言語ではないので無理
  170. 170. Q. 他の言語のバインディングは可能?
  171. 171. A. Perlと密結合しているので無理。書き直すしか </li></ul>
  172. 172. That's all Thanks for your attention!

×