How Xslate Works

  • 6,033 views
Uploaded on

My presentation "How Xslate Works" at YAPC::Asia Tokyo 2010 …

My presentation "How Xslate Works" at YAPC::Asia Tokyo 2010

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
6,033
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
17
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. How Xslate works The next generation's template engine YAPC::Asia Tokyo 2010, Day 2, 2010/10/16 10:10-10:50 Fuji, Goro (gfx) [email_address] http://github.com/gfx /
  • 2. Who am I?
    • Fuji, Goro ( 藤 吾郎 )
      • id:gfx (hatena)
      • 3. @__gfx__ (twitter)
    • I love...
      • Perl Internals
      • 4. Perl/XS
      • 5. Moose/Mouse (as Any::Moose)
  • 6. Agenda
    • What is Xslate
    • 7. How Xslate works
      • Execution process
      • 8. Why so fast
    • Futures (TODOs)
    • 9. Information
  • 10. What is Xslate?
    • A template engine for Perl5
    • 11. Written in XS
      • But also available in pure Perl (thanks to @maka2_donzoko)
    • Fast, safe, easy to enhance
    • 12. Multi-syntaxes (esp. TT-like syntax)
    • 13. Kind error messages
  • 14. What are template engines?
    • Text processor vital for web applicatins
    • 15. my $var = sprintf 'Hello, %s world', $foo; # <- this
    • 16. $template = 'Hello, %foo% world?'; # <- this
      • $templat =~ s/ %(w+)% / $vars->{$1 }/xmsge;
  • 17. So why to use Xslate?
    • There are too many template engines on CPAN
      • Template::Toolkit 2 (and ::Tiny)
      • 18. HTML::Template (and ::Pro, ::Compiled, ::JIT)
      • 19. Text::MicroTemplate (and ::Extended)
      • 20. Mojo::Template
      • 21. Text::ClearSilver (a perl binding to clearsilver)
      • 22. HTML::Mason, Text::MicroMason
      • 23. Tenjin, Template::Alloy, Template::Sandbox, etc...
    • There is no perfect module, though
  • 24. Pick up the well-known
    • Template::Toolkit 2
      • +: Rich functionality, easy to use, easy to enhance
      • 25. -: Very slow, too large spec, vulnerable to XSS
    • HTML::Template::Pro
      • +: Fast, ristricted
      • 26. -: Hard to use, poor functionality, vulnerable to XSS
    • Text::MocroTemplate
      • +: Fast enough, easy to install, tolerant to XSS
      • 27. -: Too free, only usaful to Perl programmers
  • 28. Comparison with Xslate
    • v.s. Template::Toolkit 2
      • Much faster (100+ times faster)
      • 29. Tolerant to XSS
    • v.s. HTML::Template::Pro
      • Even faster
      • 30. Allows complex expressions, and tolerant to XSS
    • v.s. Text::MocroTemplate
      • Slightly faster
      • 31. Ristricted (you cannot call system() nor fork())
  • 32. Text::ClearSilver
    • A Perl binding to clearsilver (C library)
    • 33. ClearSilver is a good template engine,
    • 34. but the official ClearSilver.pm sucks!
      • -> Text::ClearSilver (requested by id:craftworks)
    • however, ClearSilver is not perfect
  • 35. Thus, use Text::Xslate!
    • There is no the best, but is a better, Xslate
    • 36. Xslate is at least the fastest, the most safe in the template engines on CPAN
  • 37. Tutorial
    • Basic usage: Text::Xslate->new()->render()
    • 38. new() accepts: syntax (template syntax) , module (function-based modules), function (additional functions), path (include path), cache (caching level), cache_dir (used for caches)
    • 39. render() returns a rendered text; cannot output to filehandles directly, nor set calbacks (unlike TT)
    • 40. See also Text::Xslate and Text::Xslate::Manual
  • 41. Simplest example #!perl -w use 5.10.0; use strict; use Text::Xslate; my $tx = Text::Xslate-> new (); my %vars = ( lang => 'Xslate', ); # or $tx-> render ($file, vars); say $tx-> render_string (<<'EOT', vars); Hello, <: $lang :> world! EOT # -> Hello, Xslate world!
  • 42. Xslate templates <ul> :# This is a template with the default syntax Kolon :# an one-line code starts from a colon &quot;:&quot; :# comments start from a hash symbol &quot;#&quot; :# an in-line code is surrounded by ”<:” and &quot;:>&quot; : for $list -> $item { <li> <: $item.name :> = <: $item.value :> </li> : } </ul> : include 'foo.tx' { bar => 'overrided' } :# See also ` perldoc Text::Xslate::Syntax::Kolon `
  • 43. Demo
    • cpanm Text::Xslate
      • requires Any::Moose (Mouse), Data::MessagePack, etc.
    • xslate -e 'Hello, <: $ARGV[0] :>' Xslate
    • 44. xslate -s TTerse -e '[% ARGV.0 %]' 'Hi, TTerse'
    • 45. See also example files in Text-Xslate/example/
  • 46. Details
    • Performance
    • 47. Safe (XSS tolerance and ristriction)
    • 48. Multi-syntaxes
    • Enhancement
    • 51. Template cascading
  • 52. Performance (1) Run Text-Xslate/benchmark/x-rich-env.pl!
  • 53. Performance (2)
    • Sam Graham's report ”Template roundup”
      • http://www.illusori.co.uk/projects/Template-Roundup/
      • 54. based on benchmarks with Template::Benchmark
    • Accoding to this report, Xslate is fastest on the 'instance_reuse' condition
    • 55. i.e. Persistent PSGI applications will take the best performance
  • 56. Safe (1)
    • Risks caused by Template Engines
      • XSS: Cross Site Scripting
      • 57. a type of computer security vulnerability typically found in web applications that enables malicious attackers to inject client-side script into web pages viewed by other users. (by Wikipedia)
  • 58. Safe (2)
    • Typically caused by missing HTML escaping
      • e.g. [% foo | html %] (TT2)
    • Problem: TT2 requires explicit escaping
      • Explicity sucks!
  • 59. Safe (3)
    • Solution: Template engines must apply HTML-escaping automatically
    • 60. -> Text::MicroTemplate, Text::Xslate
    • 61. This is XSS tolerance
    • 62. Modern template engines (not only in Perl) should have XSS tolerance, and in fact do so.
  • 63. How to write safe templates
    • Don't use the 'raw' filter in templates
      • Don't do that: <: $foo | raw :>
      • 64. Escape $foo in perl code, because whether $foo is escaped or not is obscure from templates
      • 65. The 'raw' filter is provided only for string literals
  • 66. Multi-syntaxes
    • There are multiple template syntaxes
      • Kolon – the default, fully-featured
      • 67. TTerse – TT2 compatible
      • 68. Clevery – Smarty (in PHP) compatible
  • 69. Kolon
    • The default template syntax
    • 70. All the features are supported
    • 71. Most optimized
  • 72. TTerse
    • TT2 compatible (urged by tokuhirom++)
    • 73. Available by default
    • 74. More ristricted than the original TT2
    • 75. No plugins, exceptions, nor mysterious features
    • 76. As good as Kolon to run-time performance
    • 77. Highly compatible, but diffrent to the basics of escaping mechanism
  • 78. Clevery
    • Smarty compatible template syntax
    • 79. Need to install Text::Clevery
    • 80. With some overhead
    Clevery Clevery
  • 81. Enhancement
    • Add TT2-like methods to Xslate
      • Text::Xslate::Bridge::TT2Like
    • Function-based modules
      • Available via the &quot;module&quot; option
      • 82. No plugin namespaces are required
  • 83. Template cascading
    • Inspired by template inheritance of T::MicroTemplate::Extended (typester++)
      • Originated from Django, a framework for Python
    • More powerful than the 'include' command
    • 84. Only available in Kolon
    • 85. See also Text-Xslate/example/{cascade.pl,cascade,tx,base.tx}
  • 86. How Xslate works
    • Execution process
      • Preprocessing
      • 87. Parsing
      • 88. Compiling
        • Saveing/loading bytecode
      • Executing
    • Why so fast
  • 89. Execution Process <: $foo :> <: $foo :> print $foo; <: $foo :> fetch_s &quot;foo&quot; print end print $foo Virtual Machine { foo => 'bar' } ”bar&quot;
  • 90. Preprocessing
    • &quot; Hello, <: $lang :> world! &quot; is difficult to parse :(
    • 91. So convert to that first:
    • 92. &quot; print_raw 'Hello, '; print $lang; print_raw ' world!' &quot;
    • 93. See &T::X::Parser::preprocess
  • 94. Parsing
    • Parse &quot; print_raw 'Hello, '; print $lang; print_raw ' world!' &quot; and build syntax tree
    • 95. Using Top Down Operator Precendence method
    • 96. Explained in ” Beautiful Code ” (O'REILLY)
    • 97. &quot; It is easy to use. It feels a lot like Recursive Descent, but with the need for less code and with significantly better performance. &quot;, Douglas Crockford introduced there.
    • 98. See also http://javascript.crockford.com/tdop/
  • 99. Top Down Operator Precedence
    • A top down parsering method
    • 100. Templates -> Tokens (string) -> Symbols (T::X::Symbol) -> Nodes (T::X::Symbol)
    • 101. Symbols (e.g. '+' for infix:<+>) know what they do
    • 102. One easily can extend parsers by adding symbols
    • 103. The parser entry point is &T::X::Parser::statements
  • 104. The heart of TDOP
    • A symbol might have denotations:
      • std : statement denotation (for statements)
      • 105. nud : null denotation (for objects and prefixes)
      • 106. led : left denotation (for infixes and postfixes)
    • A &quot;denotation&quot; is a method called by statements()
  • 107. Statement Denotation
    • e.g. 'if', 'for', 'include'
    sub std_while { my($parser, $symbol) = @_; # $symbol represents 'while' my $proc = $symbol->clone( arity => 'while' # node type ); $proc->first( # set first child $parser->expression(0) ); $parser->pointy($proc); # parse return $proc; }
  • 108. Null Denotation
    • e.g. variables, literals, parens
    # '(' expr ')' sub nud_paren { my($parser, $symbol) = @_; # $symbol is '(' my $expr = $parser->expression(0); # $symbol->conterpart is ')' $parser->advance( $symbol->counterpart ); return $expr; }
  • 109. Left Denotation
    • e.g. 'if', 'for', 'include'
    sub led_infix { my($parser, $symbol, $left) = @_; return $parser->binary( $symbol, $left, $parser->expression( $symbol->lbp) ); }
  • 110. Compiling
    • Convert the abstract syntax tree into bytecode
    • 111. Bytecode is a serialized sequence of opcodes
      • # Hello, <: $lang :> world!
      • 112. print_raw_s &quot;Hello, &quot;
      • 113. fetch_s &quot;lang&quot;
      • 114. print
      • 115. print_raw_s &quot; world!&quot;
      • 116. end
    • Also perform optimization
  • 117. Saveing/Loading Bytecode
    • Precompiling is vital for template engines
      • e.g. ClearSilver compiles templates, but it cannot save compiled ones (maybe because it targets CGI)
    • Using Data::MessagePack for serialization
      • MessagePack: spec for binary data serialization
      • 118. Like JSON, but fast and small
      • 119. See also http://msgpack.org/
  • 120. Assembling
    • Convert the name of opcode to the address of it
    • 121. Using the direct threaded code paradigm
  • 122. Executing
    • Execute by the virtual machine
    • 123. An opcode (e.g. 'fetch_s') is a block of code with an argument (a sv, int, or address of opcode)
    • 124. e.g. <: $foo :> ($a is a register, $out is the reuslt)
      • fetch_s &quot;foo&quot; # $a = $vars->{foo}
      • 125. print # $out .= $a
  • 126. Execution Process <: $foo :> <: $foo :> print $foo; <: $foo :> fetch_s &quot;foo&quot; print end print $foo Virtual Machine { foo => 'bar' } ” bar&quot;
  • 127. Why so fast
    • Precompiling & optimizing
    • 128. Virtual Machine paradigm
      • Direct threaded code (but gcc only)
    • HTML escaping
    • 129. Preallocation of the output buffer
  • 130. Optimizing bytecode
    • Constant folding
      • <: 1 + 2 :> (->compile->) literal_i 3
      • 131. <: if(0) { ... } :> (->compile->) noop
    • Injection of cascading templates
      • Cascading templates are statically concatinated
      • 132. Thus, you cannot pass variables to 'cascade'
    • $ ack -i 'optimize' lib/Text/Xslate/Compiler.pm
  • 133. Virtual Machine
    • Virtual machine is fast!
    • 134. Register machine (two registers + one stack)
  • 135. Direct Threaded Code
    • Using direct pointers of labels, not pointers of functions nor indexes of function table
  • 136. HTML Escaping
    • Important for all the template engines
    • 137. Using highly optimized routines (kazuho++)
      • As fast as memcpy(3)
      • 138. tx_sv_cat_with_html_escape_force() at Text-Xslate/xs/Text-Xslate.xs
    • Experimental hacks for Text::MicroTemplate:
      • Rewriteing HTML escaping routines makes TMT faster than HTML::Template::Pro
  • 139. Preallocation
    • Assumption: a template produces a string, and the size of the string is always the same
    • 140. Then
    • 141. First: Save the size of a template
    • 142. Later: Preallocate the buffer with the saved size
  • 143. Futures
  • 148. Loop controls
    • last, next (redo?)
    • 149. <: last if $a > 10 :> in Kolon
    • 150. [% LAST IF a > 10 %] in TTerse
  • 151. Context controls
    • Xslate calls functions/methods in scalar context, but there are other cases...
    • 152. postfix:<context>?
      • <: [ $o->foo() list ] :>
    • postfix:<@context>?
      • <: [ $o->foo() @list ] :>
    • Any ideas?
  • 153. Augment Block Modifiers
    • Xslate already supports 'around', 'before', 'after'
    • 154. 'augment' like block modifiers are requested
  • 155. More templat syntaxes
    • HTML::Template (if someone wants)
    • 156. Django (in Python)
    • 157. Jinja (in Python)
    • 158. Any ideas?
  • 159. Xslate in WAF
    • There are some bridges
      • Catalyst::View::Xslate for Catalyst
      • 160. MojoX::Renderer::Xslate for Mojo
      • 161. Dancer::Template::Xslate for Dancer
      • 162. Amon uses Xslate via Tiffany
    • More examples are required!
      • Help me, hackers!
  • 163. Information
    • http://xslate.org
    • 164. Web+DB Press Vol.59
  • 165. Vote me! Mac Book Proほしいけどお金がありません!(>_<) もしよかったら投票お願いします! http://yapcasia.org/2010/vote
  • 166. Questions
    • Q. Perlコードに落とし込むという方法もあるが、なぜわざわざVMを?
    • 167. A. テンプレートのレンダリングに特化しているので速い。特にエスケープしながらの連結が効果的
    • 168. Q. XslateのVMの上でPerl6は動かせる?
    • 169. A. 普通の言語ではないので無理
    • 170. Q. 他の言語のバインディングは可能?
    • 171. A. Perlと密結合しているので無理。書き直すしか
  • 172. That's all Thanks for your attention!