Your SlideShare is downloading. ×
0
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
これからのPerlプロダクトのかたち(YAPC::Asia 2013)
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

これからのPerlプロダクトのかたち(YAPC::Asia 2013)

5,392

Published on

Published in: Technology
0 Comments
25 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,392
On Slideshare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
16
Comments
0
Likes
25
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. A brand new way of Perl product, and it‟s future Masaaki Goshima (@goccy54) mixi Inc.
  • 2. Me • Name : Masaaki Goshima (@goccy54) • Job : mixi (Tanpopo Group) – Development for Developers – Developing a platform for refining legacy software – Managing Jenkins • Personally, interested in Perl and developing yet another Perl, gperl – (YAPC::Asia 2012) Perlと出会いPerlを作る
  • 3. gperl • Fastest perl-like language – Aiming perl5 compatible syntax – Written with C++ • The last commit for repository was 11 months ago... – “Is this already departed !?”
  • 4. NO! ITS STILL ALIVE!!! • However, gperl itself has departed • Modules(lexical analyzer, parser, code generator, interpreter) are useful – Lexical analyzer: implementing perl5 syntax highlighter – Parser: static analysis tool (used as refinement of legacy perl5 code) Input ->Lexer -> Parser ->CodeGenerator -> VM (JIT) -> output
  • 5. This presentation‟s Goal • Today, we‟re going to present “Spinout projects” for each modules gperl Compiler::Lexer Compiler::Parser Compiler::CodeGenerator ::LLVM Lexer Parser CodeGenerator ???::???::??? ????? Next Module
  • 6. Agenda • Section 1 – Introduction of Compiler::* Modules • Section 2 – (Application1) Running on multi platforms • Section 3 – (Application2) Static analysis tool
  • 7. Section1 • Introduction of Compiler::* Modules 1. Compiler::Lexer – Lexical Analyzer for Perl5 2. Compiler::Parser – Create Abstract Syntax Tree for Perl5 3. Compiler::CodeGenerator::LLVM – Create LLVM IR for Perl5
  • 8. Compiler::Lexer • Lexical Analyzer for Perl5 • Features – Simple (return Array Reference of Token Object) – Fast (faster 10 ~ 100 times than PPI::Tokenize) • Wirtten with C++ • Perfect hashing for reserved keywords • memory pool for token objects – Readable Code • Nothing use parser generator like yacc/bison
  • 9. Example 1: my$v = sprintf(<<'TEMPLATE', 'yapc'); 2: %s::Asia 3: TEMPLATE 4: $v =~ s#yapc#YAPC#; %s::Asia : HereDocument TEMPLATE : HereDocumentEnd $v : Var =~ : RegOK s : RegReplace # : RegDelim yapc : RegReplaceFrom # : RegMiddleDelim YAPC : RegReplaceTo # : RegDelim ; : SemiColon my : VarDecl $v : LocalVar = : Assign sprintf : BuiltinFunc ( : LeftParenthesis << : LeftShift TEMPLATE : HereDocumentRawTag , : Comma yapc : RawString ) : RightParenthesis ; : SemiColon tokenize
  • 10. Tips • Cannot tokenize pattern 1. func*v • 「*」is Glob or Mul 2. func/ .. • 「/」is RegexDelimiter or Div 3. func<<FLAG • 「FLAG」is HereDocumentTag or Constant It may make a mistake
  • 11. Future plan • Supporting recursively tokenizing – More wide range of parsing application – Recursive parsing will take time, optimizing will be next future work func*v =>func(*v) or func() * v func/ … =>func(/../) or func() / v func<<FLAG =>func(<<FLAG) or func() << FLAG sub func($) {} or sub func() {}
  • 12. Compiler::Parser • Create Abstract Syntax Tree for Perl5 • Features – Fast (faster than PPI) – Readable Code • Simple design • Nothing use generator Can generate Virtual Machine code as walking tree by post order -> $a->{b}->[0]->c(@args) -> -> $a {} b [] c 0 @args left left left right argsright right data data
  • 13. Example my$v= sin$v + $v * $v / $v - $v&&$v my($v= (sin((($v+ (($v* $v) / $v)) - $v))&&$v)) = $v && left right rightleft The most difficult part of Perl5 parser: hidden(optional) parenthesis
  • 14. Example2) Can parse BlackPerl BEFOREHAND: close door, each window &exit; waituntiltime; open spell book; study; read (spell, $scan, select); tell us; write it, print(thehex) whileeach watches, reverse length,write again; kill spiders, pop them, chop, split, kill them. unlink arms, shift, waitand listen (listening, wait). sort the flock (then, warn"the goats", kill"the sheep"); kill them, dump qualms, shift moralities, values aside, each one; die sheep; die (to, reversethe => system you accept (reject, respect)); next step, killnext sacrifice, each sacrifice, wait, redo ritual until"all the spirits are pleased"; do it ("as they say"). do it(*everyone***must***participate***in***forbidden**s*e*x*). return last victim; package body; exitcrypt (time, times&“half a time”) &close it. select (quickly) and warnnext victim; AFTERWARDS: tell nobody. wait, waituntiltime; wait until next year, next decade; sleep, sleep, die yourself, die@last BlackPerl for Perl5
  • 15. Future plan • Replace PPI !!! – Supporting some expressions • ThreeTermOperator, Glob, given/when, goto etc.. – Supporting compatible methods PPI provides • PPI::Document::find • PPI::Document::prune
  • 16. Compiler::CodeGenerator::LLVM • Create LLVM IR for Perl5 -> $a->{b}->[0]->c(@args) -> -> $a {} b [] c 0 @args left left left right argsright right data data 1 2 3 4 5 6 7 8 9 10 Compiler::Parser Compiler::CodeGenerator::LLVM LLVM IR Running with JIT
  • 17. Native Code Other Language LLVM(Low Level Virtual Machine) • Compiler Infrastructure • Better than GNU GCC LLVM X86 ARM Power C/C++/Object ive-C JavaScript py2llvm MacRuby clang
  • 18. How to use • Dependency: clang/llvm version 3.2 or 3.3 useCompiler::Lexer; useCompiler::Parser; useCompiler::CodeGenerator::LLVM; my$tokens = Compiler::Lexer->new('')->tokenize($code); my $ast= Compiler::Parser->new->parse($tokens); my$generator = Compiler::CodeGenerator::LLVM->new(); my$llvm_ir= $generator->generate($ast); # generate LLVM IR $generator->debug_run($ast); # run with JIT
  • 19. Benchmark(Fibonacci) Language time(real)[ sec] X gperl 0.10 X81.2 LuaJIT (2.0.0-beta10) 0.12 X67.6 v8(0.8.0) 0.18 X45.1 Compiler::CodeGenerator:: LLVM 0.72 X11.2 Perl(5.16.0) 8.12 X1.0 Environment MacOSX(10.8.4) 2.2GHz Intel Core i7 Memory: 8GB N=35
  • 20. Section 2 • Application1) – Running on multi platforms
  • 21. 1. Running on Web Browser • Recently, way of writing code that runs on web browser is not onlyJavaScript – e.g.) CoffeScript, JSX, TypeScript, Dart • I wrote module for Perl5! SEE ALSO :perl.js (@gfx)
  • 22. Compiler::Tools::Transpiler • Translate Perl5 codes to JavaScript codes – (Step1) translate Perl5 codes to LLVM-IR with Compiler::* modules – (Step2) translate LLVM-IR to JavaScript codes with emscripten Perl5 LLVM emscripten
  • 23. How to use useCompiler::Tools::Transpiler; my$transpiler= Compiler::Tools::Transpiler->new({ engine=>'JavaScript‟, library_path=> [„lib‟] }); openmy $fh, '<', 'target_application.pl'; my$perl5_code = do { local$/; <$fh> }; my$javascript_code= $transpiler->transpile($perl5_code); open $fh, '>', 'target_application.js'; print$fh $javascript_code; close$fh; ※ Needs clang-3.2 and LLVM-3.2 (Not ver. 3.3 or later) because emscripten has still not support LLVM 3.3 or later
  • 24. ROADMAP/Repository • Fix some emscripten‟s bugs • Supporting accessors for HTML objects • Supporting Canvas API • etc.. • Repository – https://github.com/goccy/p5-Compiler-Tools-Transpiler.git
  • 25. 2. Running on iOS and OSX • Recently, software that can develop iOS or OSX applications using light weight language has being released – RubyMotion – mocl – Titanium • I wrote module for Perl5!
  • 26. PerlMotion iOS and OS X development using the Perl5 programming language
  • 27. PerlMotion’s Architecture UIKit Cocoa Touch Frameworks Core Animation Core Audio Core Data Perl5 x Cocoa Touch Frameworks Bindings Library LLVM IR (Links Perl5 codes and Cocoa Touch Frameworks) Perl5 Codes Compiler::Lexer Compiler::Parser Compiler::CodeGenerator::LLVM iOS Simulator iOS DeviceMacOSX Generates Native Code for each target Architecture
  • 28. DEMO • HelloWorldby PerlMotion
  • 29. Current Status Still not support functions Symbol Management System : Glob Garbage Collection Regexp Three Term Operator etc.. Not support functions Dynamic evaluation like eval, s/../../e, require Supported functions Primitive Types : Int, double, String, Array, Hash, ArrayRefence, HashReference, IOHandler, BlessedObject … Operators : binary operator, single term operator … BuiltinFunction : print, say, shift, push, sin, open … Variable Definition, Function Definition OOP System : package, bless, @ISA
  • 30. ROADMAP • Supporting deployment on device • Preparing debugging environment – (stdout/stderr/gdb..etc) • Supporting existing framework (e.g. UIKit) • Supporting CPAN modules written by Pure Perl 2014/4 Will release at April, 2014!
  • 31. Welcome your Contribution! • I want discuss design or objective – Especially, I need advice on “What Perl community likes?” (Naming rule, etc…) https://github.com/goccy/perl-motion.git
  • 32. Conclusion at Section 1 and 2 • Perl5 codes Run Anywhere!!! – iOS,OSX,Web Browser and others • Welcome your Contribution – Compiler::Lexer • https://github.com/goccy/p5-Compiler-Lexer.git – Compiler::Parser • https://github.com/goccy/p5-Compiler-Parser.git – Compiler::CodeGenerator::LLVM • https://github.com/goccy/p5-Compiler-CodeGenerator.git – Compiler::Tools::Transpiler • https://github.com/goccy/p5-Compiler-Tools-Transpiler.git – PerlMotion • https://github.com/goccy/perl-motion.git
  • 33. • Application2) – Static analysis tool Section3
  • 34. Compiler::Tools::CopyPasteDetector • Detect Copy and Paste for Perl5 • Features – Fast (detecting Engine written by C++ with POSIX Thread) – Accuracy (based on Compiler::Lexer and B::Deparse) – High functionality Visualizer (some metrics or scattergram etc…)
  • 35. DEMO • Detect Copy & Paste of Catalyst and Mojolicious
  • 36. Another Modules • Perl::MinimumVersion::Fast(@tokuhirom) – Find a minimum required version of perl for Perl code • Test::LocalFunctions::Fast(@papix) – Detect unused local function Compiler::Lexer or Compiler::Parser provide that you can write faster modules than existing module that uses PPI
  • 37. Future plan • Perl::Metrics::Simple::Fast – Will release faster module than Perl::Metrics::Simple by using Compiler::Lexer and Compiler::Parser • Next future, I will CPANize of static alnalysis module that has been used at mixi – It includes rich Visualizer
  • 38. Conclusion at Section 3 • Introduction of copy and paste detecting tool for Perl5 – Compiler::Tools::CopyPasteDetector • Compiler::Lexer or Compiler::Parser provide that you can write faster modules than existing module that uses PPI
  • 39. Finally Compiler::Lexer Compiler::Parser Compiler::CodeGenerator ::LLVM Type Inference Engine Static Typing - Coming soon - gperl will assemble these brand new modules...

×