Practical pairing of generative programming with functional programming.


Published on

The new wave of JavaScript techniques: Practical pairing of generative programming with functional programming. Eugene Lazutkin, ClubAjax, 6/4/2013

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Practical pairing of generative programming with functional programming.

  1. 1. The new wave ofJavaScript techniquesPractical pairing of generative programmingwith functional programmingEugene Lazutkin, ClubAjax, 6/4/2013Wednesday, June 5, 13
  2. 2. Generative programmingWednesday, June 5, 13
  3. 3. What is GP?Generative programming (GP):Creates source code automatically to improveprogrammer productivity.Can be considered as a form of code reuse.Sometimes called “automatic programming”.Wednesday, June 5, 13
  4. 4. Examples of GPMacro processors:C preprocessor.C++ templates and inline functions.IDE “wizards” in Eclipse, MS Visual Studio.Meta-languages and their compilers.Domain-specific languages (DSL).Wednesday, June 5, 13
  5. 5. Why do we need GP?Higher-level programming.Meta-programming.Modeling.DSL.Wednesday, June 5, 13
  6. 6. Why do we need GP?Performance improvements.Program transformations.Data transformations.Code optimization and generation.Applies to both compilers and interpreters.Wednesday, June 5, 13
  7. 7. Why do I need GP?Common narrative goes like that:“Compilers and interpreters are now verysophisticated – no need to optimize my code.”“Modern CPUs with their fancy pipelining,parallel execution, and branch predictioneliminate code inefficiencies.”Wednesday, June 5, 13
  8. 8. Don’t write inefficient codeAlways estimate the algorithmic complexity ofyour code (Big O notation).Remember simple things:Linear code usually works faster than codewith branches or loops.Calling a function is not free. If its body issmall, it may make sense to inline it.Wednesday, June 5, 13
  9. 9. Don’t write inefficient codeRemember simple things:Simplify!Simple shorter code is usually faster.Eliminate unnecessary variables.Profile!!!BTW, this was an advice by Jared Jurkiewicz.Wednesday, June 5, 13
  10. 10. Performance summaryDo not expect that an inefficient code will bemagically efficient. Always profile!In a battle of a code clarity with a codeefficiency support the clarity, if possible.Avoid premature optimization by taking complexsteps, yet do not write obviously slow code.Wednesday, June 5, 13
  11. 11. JavaScript and GPPeople coming from static languages frequentlyoverlook or discount GP facilities in JS.JS has four ways to generate code dynamically:eval(), new Function(), setTimeout(),setInterval().Browser allows to inline code.Wednesday, June 5, 13
  12. 12. JavaScript and GPAmazing code-generation facilities are a bigdifferentiator for Javascript.Yet they are frequently shunned for numerousreasons, including security, which is not appliedin all scenarios.Wednesday, June 5, 13
  13. 13. JavaScript and GPRecently GP crept in JavaScript libraries.Examples:Many templating languages are actuallycompiled in JavaScript or can be compiled:Mustache, Handlebars, jQuery templates...Wednesday, June 5, 13
  14. 14. JavaScript and GPExamples:All JavaScript-based templates:EJS, JST (e.g., Underscore, Lo-Dash)...All transpilers:CoffeeScript, GWT, Emscripten...Numerous code-level tools (minifiers...).Wednesday, June 5, 13
  15. 15. JavaScript and GPExamples:Some libraries generate code for similarfunctions from the same template in order toreduce their downloadable size, and optimizeperformance.Lo-Dash...Wednesday, June 5, 13
  16. 16. Functional programmingWednesday, June 5, 13
  17. 17. What is FP?Functional programming (FP):Treats computations as the evaluation ofmathematical functions.Avoids state, mutations, side-effects.Emphasizes “scenario” over “a list of actors”.Concerns more with “how” rather than“who” and “what”.Wednesday, June 5, 13
  18. 18. JavaScript and FP“JavaScript is Lisp in C’s clothing.” – saidDouglas Crockford back in 2001.Functions are first-class objects.Lambdas are supported (functions can beinlined, and passed around).Closures are supported (functions can accesstheir environment).Wednesday, June 5, 13
  19. 19. Practical applications of FPAbstracting control flow details with array extras:var total = data.filter(function(value){ return value % 2; }).map(function(value){ return value * value; }).reduce(function(acc, value){return acc + value;}, 0);Wednesday, June 5, 13
  20. 20. Practical applications of FPWe can even use “building blocks” now:var total = data.filter(isOdd).map(square).reduce(sum, 0);Wednesday, June 5, 13
  21. 21. Why I like this codeIt is easy to understand.It is easy to decompose or refactor.There is no trivial technical noise like names ofindex and value variables, and exit conditions.Wednesday, June 5, 13
  22. 22. Building blocksIn general we are assumed to create ourprograms from right-sized building blocks.Bigger blocks are built from smaller blocks with amortar/language.It applies to any programming paradigms.Wednesday, June 5, 13
  23. 23. ...and realityHow come in real projects we see methods andfunctions in 100s and even 1000s lines of code?Sometimes there are “good” reasons for that.It can take more time to figure out ourbuilding blocks than to code everything fromscratch.Wednesday, June 5, 13
  24. 24. ...and more realitySometimes there are “good” reasons for that.Calling myriad of small functions can bedetrimental to performance.Is it really the case? It can be verified with aprofiler, which takes time too.It is possible to create a different system of“building blocks”.Wednesday, June 5, 13
  25. 25. Case for pipesPipes are individual components of a pipeline.Each pipe has an input and an output(sometimes more than one of those).Pipeline combines individual pipes byconnecting their inputs and outputs producing aprogram.A pipeline can be used as a pipe.Wednesday, June 5, 13
  26. 26. Case for pipesUnix shells are a classic examples of pipes.Closely related concept: dataflow.Complex applications frequently can beexpressed in terms of event flow.Hot topics in JS: event streams, data streams.Wednesday, June 5, 13
  27. 27. Pipes and chainingWhile chaining is a frequent implementationtechnique for pipes, it doesn’t mean that allchaining equates to pipes.Pipes encapsulate streams of data andorchestrate processing of individual bits.Frequently chaining operates in terms oftransferred objects (streams), not data bits.Wednesday, June 5, 13
  28. 28. ChainingExamples in JS of chaining without piping:jQueryUnderscoreJavaScript array extrasAll examples above create a container objectbefore passing it down.Wednesday, June 5, 13
  29. 29. Theory and practiceWednesday, June 5, 13
  30. 30. Can we do better?Let’s use our simple example as a start point andoptimize it by inlining functions:var total = data.filter(function(value){ return value % 2; }).map(function(value){ return value * value; }).reduce(function(acc, value){return acc + value;}, 0);Wednesday, June 5, 13
  31. 31. Can we do better?var a1 = [];for(var i = 0; i < data.length; ++i){if(data[i] % 2) a1.push(data[i]);}var a2 = new Array(a1.length);for(var i = 0; i < a1.length; ++i){a2[i] = a1[i] * a1[i];}var acc = 0;for(var i = 0; i < a2.length; ++i){acc += a2[i];}var total = acc;Wednesday, June 5, 13
  32. 32. Can we do better?Let’s merge loops and eliminate intermediatearrays:var acc = 0;for(var i = 0; i < data.length; ++i){var value = data[i];if(value % 2) acc += value * value;}var total = acc;Wednesday, June 5, 13
  33. 33. Performance results0 750 1500 2250 3000msManual (38ms) Extra (2885ms)Wednesday, June 5, 13
  34. 34. We did better!And the results are in:It is faster.We loop over values manually.We cannot easily move this snippet aroundbecause our technical variable names mayclash.Ultimately it was a trade-off.Wednesday, June 5, 13
  35. 35. Can we have our cake and eat it too?Yes!Wednesday, June 5, 13
  36. 36. Can we have our cake and eat it too?We need a way to describe a result.Not how exactly get it, but its logical inference.Then we can apply GP to generate an optimalcode.Wednesday, June 5, 13
  37. 37. FP and pipesFP gives us some answers:Iterations can be described by higher-orderoperations:map, filter, fold/reduce, zipping, unzipping,partitioning.Secondary: forEach, every, some, indexOf.Wednesday, June 5, 13
  38. 38. Heya PipeWednesday, June 5, 13
  39. 39. Heya: ctr and pipeHeya is a library that provides a modernfoundation for building web applications (bothserver and client).Heya Pipe is a functional toolkit to build efficientdata and event processing pipes using GP.Heya Ctr provides a constructor to buildgenerated components.Wednesday, June 5, 13
  40. 40. The Heya wayLet’s use our simple example and optimize it withHeya Pipe:var total = data.filter(function(value){ return value % 2; }).map(function(value){ return value * value; }).reduce(function(acc, value){return acc + value;}, 0);Wednesday, June 5, 13
  41. 41. The Heya wayHere it is:var f = array(new Pipe().filter(“value % 2”).map(“value *= value;”).fold(“accumulator += value;”, 0)).compile();var total = f(data);Wednesday, June 5, 13
  42. 42. Performance results0 750 1500 2250 3000msManual (38ms) Pipe (53ms) Extra (2885ms)Wednesday, June 5, 13
  43. 43. What do we see?We are as expressive as array extras.We are as performant as manual code.The performance gap can be reduced.In any case pipes are >40x faster than arrayextras.Wednesday, June 5, 13
  44. 44. Heya Pipe conventionsHeya Pipe is modeled after array extras.It can accept the same parameters.If a string is specified instead of a function it isassumed to be a code snippet.Code snippets work by conventions.Wednesday, June 5, 13
  45. 45. Heya Pipe conventionsSnippets can have several lines of code.“Boolean” snippets assume that the last linereturns a boolean value.Snippet’s input is a “value” variable. A snippetcan modify it to return a different value.Snippets for fold-able operations can access andupdate an “accumulator” variable.Wednesday, June 5, 13
  46. 46. DebuggingPeople make mistakes ⇒ code should bedebugged.How to debug a generated code?What is Heya Pipe story here?Wednesday, June 5, 13
  47. 47. DebuggingThe screenshot shows node.js debugging withnode-inspector.The generated code appears in the file panel.The code is fully readable.Wednesday, June 5, 13
  48. 48. Heya PipeProvides a rich foundation for pipes:forEach, transform, map, filter, indexOf, every,some, fold, scan, skip, take, skipWhile,takeWhile, voidResult.Can work with potentially infinite streams.Wednesday, June 5, 13
  49. 49. Heya PipeCan iterate over numerous containers:Array (sparse and dense, including in reverse),array slices (including in reverse), Object (keys,values, own and inherited), iterators,enumerators, generators, and perform unfold.Includes a compiler, and an interpreter.Can use pipes as sub-pipes.Wednesday, June 5, 13
  50. 50. Heya CtrImplements GP helpers.Includes a simple formatting-awaretemplating.Allows managing closures.Serves as a foundation for Heya Pipes.Wednesday, June 5, 13
  51. 51. Closures with Heya CtrAllows to inject variables in a closure, which willbe directly accessible from snippets by name.Does not use “with” statement because it isincompatible with strict mode.Fully strict mode compatible.Allows to access and modify closure-boundvariables.Wednesday, June 5, 13
  52. 52. !at’s allFolks!Wednesday, June 5, 13