これからのPerlプロダクトのかたち(YAPC::Asia 2013)
Upcoming SlideShare
Loading in...5
×
 

これからのPerlプロダクトのかたち(YAPC::Asia 2013)

on

  • 5,511 views

 

Statistics

Views

Total Views
5,511
Views on SlideShare
4,377
Embed Views
1,134

Actions

Likes
27
Downloads
14
Comments
0

9 Embeds 1,134

http://takiguchi0817.github.io 588
http://goccy54.hatenablog.com 326
https://twitter.com 93
http://rejasupotaro.github.io 87
http://localhost 22
http://cloud.feedly.com 10
http://www.plurk.com 6
http://summary 1
http://news.google.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

これからのPerlプロダクトのかたち(YAPC::Asia 2013) これからのPerlプロダクトのかたち(YAPC::Asia 2013) Presentation Transcript

  • A brand new way of Perl product, and it‟s future Masaaki Goshima (@goccy54) mixi Inc.
  • Me • Name : Masaaki Goshima (@goccy54) • Job : mixi (Tanpopo Group) – Development for Developers – Developing a platform for refining legacy software – Managing Jenkins • Personally, interested in Perl and developing yet another Perl, gperl – (YAPC::Asia 2012) Perlと出会いPerlを作る
  • gperl • Fastest perl-like language – Aiming perl5 compatible syntax – Written with C++ • The last commit for repository was 11 months ago... – “Is this already departed !?”
  • NO! ITS STILL ALIVE!!! • However, gperl itself has departed • Modules(lexical analyzer, parser, code generator, interpreter) are useful – Lexical analyzer: implementing perl5 syntax highlighter – Parser: static analysis tool (used as refinement of legacy perl5 code) Input ->Lexer -> Parser ->CodeGenerator -> VM (JIT) -> output
  • This presentation‟s Goal • Today, we‟re going to present “Spinout projects” for each modules gperl Compiler::Lexer Compiler::Parser Compiler::CodeGenerator ::LLVM Lexer Parser CodeGenerator ???::???::??? ????? Next Module
  • Agenda • Section 1 – Introduction of Compiler::* Modules • Section 2 – (Application1) Running on multi platforms • Section 3 – (Application2) Static analysis tool
  • Section1 • Introduction of Compiler::* Modules 1. Compiler::Lexer – Lexical Analyzer for Perl5 2. Compiler::Parser – Create Abstract Syntax Tree for Perl5 3. Compiler::CodeGenerator::LLVM – Create LLVM IR for Perl5
  • Compiler::Lexer • Lexical Analyzer for Perl5 • Features – Simple (return Array Reference of Token Object) – Fast (faster 10 ~ 100 times than PPI::Tokenize) • Wirtten with C++ • Perfect hashing for reserved keywords • memory pool for token objects – Readable Code • Nothing use parser generator like yacc/bison
  • Example 1: my$v = sprintf(<<'TEMPLATE', 'yapc'); 2: %s::Asia 3: TEMPLATE 4: $v =~ s#yapc#YAPC#; %s::Asia : HereDocument TEMPLATE : HereDocumentEnd $v : Var =~ : RegOK s : RegReplace # : RegDelim yapc : RegReplaceFrom # : RegMiddleDelim YAPC : RegReplaceTo # : RegDelim ; : SemiColon my : VarDecl $v : LocalVar = : Assign sprintf : BuiltinFunc ( : LeftParenthesis << : LeftShift TEMPLATE : HereDocumentRawTag , : Comma yapc : RawString ) : RightParenthesis ; : SemiColon tokenize
  • Tips • Cannot tokenize pattern 1. func*v • 「*」is Glob or Mul 2. func/ .. • 「/」is RegexDelimiter or Div 3. func<<FLAG • 「FLAG」is HereDocumentTag or Constant It may make a mistake
  • Future plan • Supporting recursively tokenizing – More wide range of parsing application – Recursive parsing will take time, optimizing will be next future work func*v =>func(*v) or func() * v func/ … =>func(/../) or func() / v func<<FLAG =>func(<<FLAG) or func() << FLAG sub func($) {} or sub func() {}
  • Compiler::Parser • Create Abstract Syntax Tree for Perl5 • Features – Fast (faster than PPI) – Readable Code • Simple design • Nothing use generator Can generate Virtual Machine code as walking tree by post order -> $a->{b}->[0]->c(@args) -> -> $a {} b [] c 0 @args left left left right argsright right data data
  • Example my$v= sin$v + $v * $v / $v - $v&&$v my($v= (sin((($v+ (($v* $v) / $v)) - $v))&&$v)) = $v && left right rightleft The most difficult part of Perl5 parser: hidden(optional) parenthesis
  • Example2) Can parse BlackPerl BEFOREHAND: close door, each window &exit; waituntiltime; open spell book; study; read (spell, $scan, select); tell us; write it, print(thehex) whileeach watches, reverse length,write again; kill spiders, pop them, chop, split, kill them. unlink arms, shift, waitand listen (listening, wait). sort the flock (then, warn"the goats", kill"the sheep"); kill them, dump qualms, shift moralities, values aside, each one; die sheep; die (to, reversethe => system you accept (reject, respect)); next step, killnext sacrifice, each sacrifice, wait, redo ritual until"all the spirits are pleased"; do it ("as they say"). do it(*everyone***must***participate***in***forbidden**s*e*x*). return last victim; package body; exitcrypt (time, times&“half a time”) &close it. select (quickly) and warnnext victim; AFTERWARDS: tell nobody. wait, waituntiltime; wait until next year, next decade; sleep, sleep, die yourself, die@last BlackPerl for Perl5
  • Future plan • Replace PPI !!! – Supporting some expressions • ThreeTermOperator, Glob, given/when, goto etc.. – Supporting compatible methods PPI provides • PPI::Document::find • PPI::Document::prune
  • Compiler::CodeGenerator::LLVM • Create LLVM IR for Perl5 -> $a->{b}->[0]->c(@args) -> -> $a {} b [] c 0 @args left left left right argsright right data data 1 2 3 4 5 6 7 8 9 10 Compiler::Parser Compiler::CodeGenerator::LLVM LLVM IR Running with JIT
  • Native Code Other Language LLVM(Low Level Virtual Machine) • Compiler Infrastructure • Better than GNU GCC LLVM X86 ARM Power C/C++/Object ive-C JavaScript py2llvm MacRuby clang
  • How to use • Dependency: clang/llvm version 3.2 or 3.3 useCompiler::Lexer; useCompiler::Parser; useCompiler::CodeGenerator::LLVM; my$tokens = Compiler::Lexer->new('')->tokenize($code); my $ast= Compiler::Parser->new->parse($tokens); my$generator = Compiler::CodeGenerator::LLVM->new(); my$llvm_ir= $generator->generate($ast); # generate LLVM IR $generator->debug_run($ast); # run with JIT
  • Benchmark(Fibonacci) Language time(real)[ sec] X gperl 0.10 X81.2 LuaJIT (2.0.0-beta10) 0.12 X67.6 v8(0.8.0) 0.18 X45.1 Compiler::CodeGenerator:: LLVM 0.72 X11.2 Perl(5.16.0) 8.12 X1.0 Environment MacOSX(10.8.4) 2.2GHz Intel Core i7 Memory: 8GB N=35
  • Section 2 • Application1) – Running on multi platforms
  • 1. Running on Web Browser • Recently, way of writing code that runs on web browser is not onlyJavaScript – e.g.) CoffeScript, JSX, TypeScript, Dart • I wrote module for Perl5! SEE ALSO :perl.js (@gfx)
  • Compiler::Tools::Transpiler • Translate Perl5 codes to JavaScript codes – (Step1) translate Perl5 codes to LLVM-IR with Compiler::* modules – (Step2) translate LLVM-IR to JavaScript codes with emscripten Perl5 LLVM emscripten
  • How to use useCompiler::Tools::Transpiler; my$transpiler= Compiler::Tools::Transpiler->new({ engine=>'JavaScript‟, library_path=> [„lib‟] }); openmy $fh, '<', 'target_application.pl'; my$perl5_code = do { local$/; <$fh> }; my$javascript_code= $transpiler->transpile($perl5_code); open $fh, '>', 'target_application.js'; print$fh $javascript_code; close$fh; ※ Needs clang-3.2 and LLVM-3.2 (Not ver. 3.3 or later) because emscripten has still not support LLVM 3.3 or later
  • ROADMAP/Repository • Fix some emscripten‟s bugs • Supporting accessors for HTML objects • Supporting Canvas API • etc.. • Repository – https://github.com/goccy/p5-Compiler-Tools-Transpiler.git
  • 2. Running on iOS and OSX • Recently, software that can develop iOS or OSX applications using light weight language has being released – RubyMotion – mocl – Titanium • I wrote module for Perl5!
  • PerlMotion iOS and OS X development using the Perl5 programming language
  • PerlMotion’s Architecture UIKit Cocoa Touch Frameworks Core Animation Core Audio Core Data Perl5 x Cocoa Touch Frameworks Bindings Library LLVM IR (Links Perl5 codes and Cocoa Touch Frameworks) Perl5 Codes Compiler::Lexer Compiler::Parser Compiler::CodeGenerator::LLVM iOS Simulator iOS DeviceMacOSX Generates Native Code for each target Architecture
  • DEMO • HelloWorldby PerlMotion
  • Current Status Still not support functions Symbol Management System : Glob Garbage Collection Regexp Three Term Operator etc.. Not support functions Dynamic evaluation like eval, s/../../e, require Supported functions Primitive Types : Int, double, String, Array, Hash, ArrayRefence, HashReference, IOHandler, BlessedObject … Operators : binary operator, single term operator … BuiltinFunction : print, say, shift, push, sin, open … Variable Definition, Function Definition OOP System : package, bless, @ISA
  • ROADMAP • Supporting deployment on device • Preparing debugging environment – (stdout/stderr/gdb..etc) • Supporting existing framework (e.g. UIKit) • Supporting CPAN modules written by Pure Perl 2014/4 Will release at April, 2014!
  • Welcome your Contribution! • I want discuss design or objective – Especially, I need advice on “What Perl community likes?” (Naming rule, etc…) https://github.com/goccy/perl-motion.git
  • Conclusion at Section 1 and 2 • Perl5 codes Run Anywhere!!! – iOS,OSX,Web Browser and others • Welcome your Contribution – Compiler::Lexer • https://github.com/goccy/p5-Compiler-Lexer.git – Compiler::Parser • https://github.com/goccy/p5-Compiler-Parser.git – Compiler::CodeGenerator::LLVM • https://github.com/goccy/p5-Compiler-CodeGenerator.git – Compiler::Tools::Transpiler • https://github.com/goccy/p5-Compiler-Tools-Transpiler.git – PerlMotion • https://github.com/goccy/perl-motion.git
  • • Application2) – Static analysis tool Section3
  • Compiler::Tools::CopyPasteDetector • Detect Copy and Paste for Perl5 • Features – Fast (detecting Engine written by C++ with POSIX Thread) – Accuracy (based on Compiler::Lexer and B::Deparse) – High functionality Visualizer (some metrics or scattergram etc…)
  • DEMO • Detect Copy & Paste of Catalyst and Mojolicious
  • Another Modules • Perl::MinimumVersion::Fast(@tokuhirom) – Find a minimum required version of perl for Perl code • Test::LocalFunctions::Fast(@papix) – Detect unused local function Compiler::Lexer or Compiler::Parser provide that you can write faster modules than existing module that uses PPI
  • Future plan • Perl::Metrics::Simple::Fast – Will release faster module than Perl::Metrics::Simple by using Compiler::Lexer and Compiler::Parser • Next future, I will CPANize of static alnalysis module that has been used at mixi – It includes rich Visualizer
  • Conclusion at Section 3 • Introduction of copy and paste detecting tool for Perl5 – Compiler::Tools::CopyPasteDetector • Compiler::Lexer or Compiler::Parser provide that you can write faster modules than existing module that uses PPI
  • Finally Compiler::Lexer Compiler::Parser Compiler::CodeGenerator ::LLVM Type Inference Engine Static Typing - Coming soon - gperl will assemble these brand new modules...