Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Code Generation in Perl


Published on

"Code Generation in Perl" presented to Silicon Valley Perl in Santa Clara CA by Ian Kluft on September 7, 2017. The presentation surveys the landscape of code generation tools and techniques including preprocessers, compliers and model languages, as well as CPAN modules to help with those tasks. Code generation has always been useful for automation of tedious tasks. There is also some debunking of myths from science fiction about what is currently possible with code generation. No matter how much processing is involved, at some level someone still has to write the programs.

Published in: Software
  • Be the first to comment

Code Generation in Perl

  1. 1. Code Generation in Perl Presentation by Ian Kluft for Silicon Valley Perl September 7, 2017 Santa Clara, California
  2. 2. What is Code Generation? ● Programs that write programs ● There are many ways to approach this ● Advantage: lets the coder work at a higher level ● Disadvantage: someone still has to write program
  3. 3. The simplest example ● A program hard-coded to print another program ● Even the simple example has non-trivial quoting #!/usr/bin/perl use strict; use warnings; print qq(#!/usr/bin/perln); print qq(use strict;n); print qq(print "hello worldn";n);
  4. 4. Other approaches ● Templates – fill in the blanks ● Translators – parse, analyze and re-generate ● Compilers – parse high level, generate lower level ● SQL schemas & queries – generate SQL code ● XML parsing – generate programs from XML data ● UML tools – draw diagrams, generate code
  5. 5. CPAN Modules which generate code ● At a low level it’s widespread – Many modules & scripts generate code for eval() ● Perl build tools ● SQL generation for queries and schemas ● Perl application frameworks – Easier to get started with tools that generate examples ● Compilers
  6. 6. CPAN Modules Perl build tools ● CPAN Modules which generate code – Dist::Zilla – ExtUtils::MakeMaker – Module::Build ● These generate build scripts from module configuration data
  7. 7. CPAN Modules SQL code generation ● DBIx::Class – object relational mapper ● Fey – SQL schema and query generator ● …CPAN is huge - the list goes on and on... ● Many examples – which do you like?
  8. 8. CPAN Modules XML & data code generators ● Data::Dumper – dumps data as Perl code ● Faker – extensible fake data generator ● Not as much code generation from XML on CPAN – Obvious category for ad hoc tools – Any XML parser can be used to generate code ● Writing HTML is not technically writing code – But it also has similarities
  9. 9. CPAN Modules Compilers ● B::* - access to Perl compiler ● Compiler generation tools – Parse::Yapp/Parse::Eyapp – compiler parser generator ● Similar to Unix “yacc” but generates Perl code – Parse::RecDescent – generate recursive descent compiler – Parse::Lex – generate lexical analyzer for compiler ● A lot of CPAN modules claiming to “compile” actually “translate”
  10. 10. How Compilers Work ● High level view of a compiler Preprocessor Scanner/ Tokenizer Parser Semantics ● Preprocessor – macros & text manipulation ● Scanner – generates stream of tokens – Removes comments ● Parser – recognizes language structures ● Semantics – generate code – Some compilers like Perl make intermediate code for interpretation
  11. 11. Using Parser Generator Tools ● Have you noticed that language compilers do things that are way above the capabilities of regular expressions? – RegEx’es are ideal for tokenizers/scanners – There’s another level of language parsing above that ● One example is to grab a yacc grammar and translate it to Parse::Yapp or Parse::Eyapp – I did that once for a Perl script that parsed /etc/sudoers files for automated security audits ● Language definitions in Backus-Naur Form (BNF) can be converted to language parser defintions
  12. 12. UML Unified Modeling Language ● XML format representing standard diagrams – Parts of a program or system design – Analysis of code can generate UML for documentation – Active research into ways to generate code from UML ● Code generation can be done ● But diagramming is far more tedious than typing code ● You can experiment with this on your own projects ● Idea is to use UML as design/documentation & generate code
  13. 13. UML – Unified Modeling Language types of diagrams Diagram Class Diagram Component Diagram Object Diagram Activity Diagram Use Case Diagram Profile Diagram Composite Structure Diagram Deployment Diagram Package Diagram Interaction Diagram State Machine Diagram Sequence Diagram Communication Diagram Interaction Overview Diagram Timing Diagram Notation: UML Behaviour Diagram Structure Diagram UML diagram types – as a class diagram (from Wikipedia)
  14. 14. UML – Unified Modeling Language example class: Pizza UML code generation makes class source code starter templates for ● Pizza ● CrustEnum ● ToppingList ● ToppingEnum ● SizeEnum How would you go from here to write the code for the classes in UML? Good ways to do that are still open to suggestion.
  15. 15. UML – Unified Modeling Language Tools for Code Generation from UML ● Open source “umbrello” from KDE generated previous slide’s graphic ● Eclipse IDE has UML generation tools ● CPAN has many UML modules, mostly for generating UML from Perl for analysis ● Many other examples...
  16. 16. Setting expectations What a non-engineer manager wants… This one doesn’t grasp the current limitations. He isn’t alone. What people want is often a hint where research should push the limits for the future.
  17. 17. Setting expectations ”Artificial Intelligence” ● Artificial Intelligence has a broken promise built right into its name ● It has always been advanced algorithms – It’s always non-trivial – Not all uses of the term AI are as sophisticated as claimed ● Its results usually appear clever ● It can do lots of things people don’t do well ● But it still can’t think ● It isn’t real or artificial intelligence… yet
  18. 18. Science Fiction examples Code generation and AI ● One early SciFi story: “The Adolescence of P1” by Thomas J Ryan, 1977 ● Based on modem-connected IBM 360 mainframes ● Program to break OS “supervisor” – Needed decision trees, code generator, threat analysis – When told to shut down it played dead, kept running – AI evolved organically – Installed itself like a worm, broke OS like a virus ● Few SciFi AI’s attempted such depth
  19. 19. Code Generation Setting expectations back to reality ● SciFi myths need to be debunked ● Code generation just adds a level of abstraction ● Someone still has to write the code ● Short of an actual artificial intelligence, programs won’t be writing programs any time soon ● Maybe during 21st Century? – 20th Century showed how much advancement can happen – But AI is a fundamentally difficult computing challenge – Don’t count on it
  20. 20. Code Generation Q & A