Ast transformation


Published on

Groovy's AST Transformation is a compile time meta-programming technique and allows developer to hook into compilation process and add new fields or methods, or modify existing methods.

Published in: Technology, Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Ast transformation

  1. 1. Meta-Programming with ASTTransformations in GroovyGagan AgrawalXebia
  2. 2. AgendaMeta-Programming with AST TransformationWhat ?How ?Why ?
  3. 3. Metaprogramming
  4. 4. What is MetaprogrammingIs the writing of computer programs that write ormanipulate other programsMinimizes the number of lines of code to expressa solutionReduces development timeCould be Run-time or Compile-timeGroovyRun-Time : MetaClassCompile-Time : AST Transformations
  5. 5. Concrete Syntax Tree
  6. 6. Concrete Syntax TreeIs a representation of grammar in a tree-like form.Is a one-to-one mapping from the grammar to atree-form.
  7. 7. Concrete Syntax TreeExample in Creturn a+2;
  8. 8. Abstract Syntax Tree
  9. 9. Abstract Syntax TreeIs a tree representation of the abstract syntacticstructure of source codeEach node of the tree denotes a construct insource codeAbstract – Does not represent every detail thatappears in the real syntaxE.g.Parentheses are implicitSemicolon ; is discarded
  10. 10. Abstract Syntax Treereturn a+2;
  11. 11. AST Transformation
  12. 12. AST TransformationsCompile-time metaprogrammingAllows developers to hook into the compilationprocessTo modify Abstract Syntax Tree before bytecodegenerationPerformance is better than Run-timemetaprogramming
  13. 13. AST TransformationCompiler Phases Initialization : Source files are opened and environment configured Parsing : The grammar is used to produce tree of tokens representing the source code Conversion : An Abstract Syntax Tree(AST) is created from token trees Semantic Analysis : Performs consistency and validity checks that the grammar cant check for and resolves classes
  14. 14. AST TransformationCompiler Phases : Canonicalization: Complete the AST Instruction Selection : instruction set is choosen, e.g. Java5 or pre Java5 Class Generation : creates binary output in memory Output : Write the binary output to the file system Finalization : Perform any last cleanup
  15. 15. Expressions, Statements and Blocks
  16. 16. ExpressionsAn expression is a construct made up of variables,operators and method invocationsEvaluates to a single valueE.g. 2+3 evaluates to 5A variable is an expression because it denotes avalue in memory int var = 0;
  17. 17. Expression Compound Expression 1*2*3 x+y/100 (x+y)/100 Other Types Cast Expression - (int c = (int)d) Constant Expression – (null, true, false) Field Expression – ( Method Call Expression – (object.method()) Boolean Expression - (value1==value2)
  18. 18. StatementsRoughly equivalent to sentences in naturallanguages.Forms a complete unit of execution.TypesExpression StatementsDeclaration StatementsControl Flow Statements
  19. 19. Expression StatementsCreated by terminating expression withsemicolon (;)Assignment StatementaValue = 8933.234;Increment StatementaValue++;Method Invocation StatementSystem.out.println("Hello World!");Object Creation StatementBicycle myBike = new Bicycle();
  20. 20. Declaration and Control flow StatementsDeclaration Statementsdouble aValue = 8933.234;Control flow StatementsIf-then and if-then-else StatementsSwitch StatementsWhile and do-while StatementsFor Statements
  21. 21. Block StatementsA block is a group of zero or more statementsbetween balanced bracesCan be used anywhere a single statement isallowed.
  22. 22. Block Statementsclass BlockDemo { public static void main(String[] args) { boolean condition = true; if (condition) { // begin block 1 System.out.println("Condition is true."); } // end block one else { // begin block 2 System.out.println("Condition is false."); } // end block 2 } }
  23. 23. AST Building Blocks
  24. 24. AST Building BlocksAST can be transformed in one of the two waysCore AST APIsASTBuilder
  25. 25. Core AST APIs
  26. 26. Core AST APIs A S T No d e A n n o ta te d No d e G e n e r ic s T y p e M o d u le N o d e Sta te m e nt E x p r e s s io n
  27. 27. Core AST APIsASTNodeBase class for any AST node.This class supports basic information used in all nodes ofthe ASTLine and column number information.Meta data of a nodeA phase operation or transform can use this to transport arbitraryinformation to another phase operation or transform providedtransform runs after the part storing the information.
  28. 28. Core AST APIs - ExpressionExpressionBooleanExpressionCastExpressionConstantExpressionConstructorCallExpressionFieldExpressionMethodCallExpressionStaticMethodCallExpressionVariableExpression
  29. 29. Core AST APIs - Expression this.println(message) can be represented as- new MethodCallExpression( new VariableExpression("this"), new ConstantExpression("println"), new ArgumentListExpression( new ConstantExpression(message) ) )
  30. 30. Core AST API - Statement StatementBlockStatementSwitchStatementCaseStatementBreakStatementExpressionStatementIfStatementTryCatchStatementWhileStatementReturnStatement
  31. 31. Core AST API - Statement this.println(message) can be represented as- return new ExpressionStatement( new MethodCallExpression( new VariableExpression("this"), new ConstantExpression("println"), new ArgumentListExpression( new ConstantExpression(message) ) ) )
  32. 32. AST Builder
  33. 33. AST BuilderThree ways to build AST via ASTBuilderBuild From StringsBuild From CodeBuild From a DSL-like Specification
  34. 34. Building AST From String def builder = new AstBuilder() List<ASTNode> nodes = builder.buildFromString( CompilePhase.SEMANTIC_ANALYSIS, true, """ this.println(Hello World) """ )
  35. 35. Building AST From StringPhase parameter tells AST from which phase toreturn AST. "statementsOnly" boolean parameter tells thebuilder to discard the generated top level ScriptClassNode. String paramter is the input The builder returns List<ASTNode>
  36. 36. Building AST From StringProduces following ASTBlockStatement-> ExpressionStatement -> MethodCallExpression -> VariableExpression -> ConstantExpression -> ArgumentListExpression -> ConstantExpression
  37. 37. Building AST From StringAdvantagesDoes not require author to understand ASTNode subtypesAllows author to target a CompilePhaseCommunicates source code being generatedRobust - Should need no changes even if AST is updated inrelease
  38. 38. Building AST From StringDisadvantagesIDE can not check syntax or grammarIDE can not refactor across StringSome entities cannot be created, like the AST for a fielddeclaration
  39. 39. Building AST From Code def builder = new AstBuilder() List<ASTNode> nodes = builder.buildFromCode( CompilePhase.SEMANTIC_ANALYSIS, true){ println "Hello World" }
  40. 40. Building AST From CodeProduces following AST BlockStatement -> ExpressionStatement -> MethodCallExpression -> VariableExpression -> ConstantExpression -> ArgumentListExpression -> ConstantExpression
  41. 41. Building AST From CodeAdvantagesClearly communicates source being generatedDoes not require author to understand ASTNode subtypesAllows author to target a CompilePhaseRobust - Should need no changes even if AST is updated ina releaseIDE supports syntax checking and refactoring in Closure
  42. 42. Building AST From CodeDisadvantages Some entities cannot be created, like the AST for a fielddeclaration
  43. 43. Building AST From DSL-like Specification List<ASTNode> nodes = new AstBuilder().buildFromSpec{ block{ expression { methodCall { variable this constant println argumentList { if(locale == US) constant Hello if(locale == FR) constant Bonjour else constant "Ni hao" } } } } }
  44. 44. Building AST From DSL-like SpecificationAdvantagesAllows conditionals (or any Groovy code) to be executedduring the AST building process.Allows any ASTNode subtype to be createdFully documented with lengthy examples in TestCase
  45. 45. Building AST From DSL-like SpecificationDisadvantagesIt can be difficult to determine what AST you need towriteVerbose - does not always communicate the source beingcreatedFragile - AST may need to change between major releasesIDE does not yet provide code tips
  46. 46. AST TransformationTwo types of AST TransformationsLocal TransformationsGlobal Transformations
  47. 47. Local AST TransformationsCan be applied on a per class basis.Can only be applied at semantic analysis or laterphasesAnnotation Driven@Retention - Retention Policy should be SOURCE@Target – Can be Method, Field, Constructor ..//need toverify@GroovyASTTransformationClass
  48. 48. Local AST TransformationsLogging Example @Retention(RetentionPolicy.SOURCE) @Target([ElementType.METHOD]) @GroovyASTTransformationClass("LoggingASTTrans formation") @interface WithLogging { }
  49. 49. Local AST Transformations @GroovyASTTransformation(phase=CompilePhase. SEMANTIC_ANALYSIS) class LoggingASTTransformation implements ASTTransformation{ @Override public void visit(ASTNode[] nodes, SourceUnit source) { ..... } }
  50. 50. Local AST TransformationsUsage class TestAst { @WithLogging void testMethod(){ println "This is test Method" } }
  51. 51. Global AST TransformationsGlobal transformations are applied to by thecompiler on the code being compiled, whereverthe transformation apply.A JAR should be added to the classpath of thecompilerIt should contain a service locator file at META-INF/services/org.codehaus.groovy.transform.ASTTransformationFile should have a line with the name of thetransformation class
  52. 52. Global AST TransformationsThe transformation class must have a no-argsconstructor and implement theorg.codehaus.groovy.transform.ASTTransformation interfaceIt will be run against every source in thecompilation
  53. 53. Global AST Transformation Example
  54. 54. Global AST TransformationSummaryWrite an ASTTransformation subclassCreate a Jar metadata file containing the name of yourASTTransformationCreate a Jar containing the class and metadataInvoke groovyc with that Jar on your classpath.
  55. 55. Built-in Annotations
  56. 56. @DelegateWith @Delegate annotation, a class field becomesan object to which method calls are delegated.Example class Event{ @Delegate Date when String title, url }
  57. 57. @Delegate Groovy compiler adds all of Dates methods to theEvent class. Behind the scene methods simply delegate thecall to Date field.
  58. 58. @Singleton class T { private static volatile T instance private T() {} static T getInstance () { if (instance) { instance } else { synchronized(T) { if (instance) { instance } else { instance = new T () } } } } }
  59. 59. @Singleton @Singleton class T { //properties... //methods.. }
  60. 60. @ImmutableNo mutators (methods that modify internal state)Class must be finalFields must be private and finalequals, hashCode and toString must beimplemented in terms of the fields if you want tocompare your objects or use them as keys in e.g.maps
  61. 61. @ImmutableJava code requires lot of boiler plate code.Groovy requires only annotation @Immuatable onclass @Immutable final class Person { String first String last }
  62. 62. @PackageScope class PackageScopeExample { @PackageScope String name }
  63. 63. @Lazy class LazyExample { @Lazy pets = [Cat, Dog, Bird] static main(args){ def example = new LazyExample() println "Pets intialized : "+example.dump().contains(Cat) println "List size : "+example.pets.size() println "Pets intialized : "+example.dump().contains(Cat) } }
  64. 64. @NewifyRuby Approach @Newify([Integer]) void test(){ println }
  65. 65. @NewifyPython Approach class Tree { def elements Tree(Object... elements) { this.elements = elements as List } } class Leaf { def value Leaf(value) { this.value = value } }
  66. 66. @Newify def buildTree() { new Tree(new Tree(new Leaf(1), new Leaf(2)), new Leaf(3)) } @Newify([Tree, Leaf])def buildTree() { Tree(Tree(Leaf(1), Leaf(2)), Leaf(3)) }
  67. 67. @Category interface Vehicle{ String getName() } @Category(Vehicle) class FlyingAbility{ def fly(){ "I am ${name} and I fly" } }
  68. 68. @Mixin @Category(Vehicle) class DrivingAbility{ def drive(){ "I am ${name} and I drive" } } @Mixin([DrivingAbility, FlyingAbility]) class JamesBondVehicle implements Vehicle{ public String getName(){ "James Bond Vehicle" } }
  69. 69. Where AST Transformations are used?Grails 2.0GormTransformerControllerActionTransformerControllerTransformerGormToJpaTransformerSpock frameworkGrails PluginsAudit-Trail PluginRich Domain Classes PluginRedis PluginNeo4j Plugin
  70. 70. Thank You