Open Source Compiler Construction for the JVM

1,998 views

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,998
On SlideShare
0
From Embeds
0
Number of Embeds
71
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Open Source Compiler Construction for the JVM

  1. 1. Open Source Compiler Construction (for the JVM) Tom Lee Shine Technologies OSCON Java 2011
  2. 2. Overview <ul><li>About the Java Virtual Machine
  3. 3. About Scala
  4. 4. About Apache BCEL
  5. 5. A Generalized Compiler Architecture
  6. 6. Introducing “Awesome”
  7. 7. Writing a Stub Code Generator for “Awesome”
  8. 8. Abstract Syntax Trees (ASTs)
  9. 9. Parsing “Awesome” with Scala
  10. 10. What Next? </li></ul>
  11. 11. About the Java Virtual Machine <ul><li>Stack-based machine. </li><ul><li>Operands pushed onto a stack, operated upon by opcodes. </li></ul><li>Heavily influenced by “Java The Language”.
  12. 12. Allows interop between various source languages.
  13. 13. JVM bytecode is architecture-independent.
  14. 14. Multiple implementations. </li><ul><li>Oracle / HotSpot
  15. 15. Apache / Harmony
  16. 16. OpenJDK
  17. 17. Many more... </li></ul></ul>
  18. 18. About Scala <ul><li>http://www.scala-lang.org/
  19. 19. Functional/OOP hybrid programming language </li><ul><li>First class functions
  20. 20. Pattern matching
  21. 21. Classes & traits
  22. 22. Etc. </li></ul><li>Runs on the JVM. </li><ul><li>And .NET. </li></ul><li>Own standard library (on top of the Java standard library). </li></ul>
  23. 23. About Apache BCEL <ul><li>http://jakarta.apache.org/bcel/
  24. 24. “Bytecode Engineering Library”
  25. 25. Emit JVM bytecode with a reasonably straightforward API.
  26. 26. Also supports reading class files, modifying classes, etc. </li><ul><li>But we won't use that stuff here. </li></ul></ul>
  27. 27. A Generalized Compiler Architecture <ul><li>Scanner matches patterns in source code, outputs tokens.
  28. 28. Parser organizes tokens into an Abstract Syntax Tree (AST). </li><ul><li>Or a Parse Tree . </li></ul><li>Semantic checks or optimizations of the AST may occur here.
  29. 29. Code Generator traverses the AST to produce target code. </li></ul>
  30. 30. Introducing “Awesome” program ::= ( expression ';')* expression ::= sum sum ::= product (('+' | '-') product )* product ::= number (('*' | '/') number )* number ::= /[0-9]+/ The “Awesome” compiler will generate JVM bytecode to display each parsed expression. (Awesome, eh?)
  31. 31. Writing a Stub Code Generator for “Awesome” <ul><li>Work backwards and write a stub code generator first . </li><ul><li>Immediate feedback!
  32. 32. A solid foundation on which to build the rest of the compiler. </li></ul><li>Generate a “Hello World” class file with BCEL .
  33. 33. We'll make the code generator do “real” stuff later. </li></ul>
  34. 34. Abstract Syntax Trees (ASTs) <ul><li>A logical, in-memory representation of the source program.
  35. 35. Constructed by the parser.
  36. 36. Semantic checks and optimizations possible at the AST level. </li><ul><li>Outside the scope of this presentation – sorry! </li></ul><li>Let's add the beginnings of an AST to our compiler. </li></ul>
  37. 37. Parsing “Awesome” with Scala <ul><li>Use parser combinators . </li><ul><li>Combine small parsing functions together to describe a language.
  38. 38. Scanner and parser wrt the Generalized Compiler Architecture. </li></ul><li>Describe languages using something of a pseudo-EBNF.
  39. 39. You can write your own parsers too. </li><ul><li>e.g. for string literals. </li></ul><li>We'll use built-in parsers to identify integers & semi-colons. </li></ul>
  40. 40. The “Awesome” Grammar Revisited program ::= ( expression ';')* expression ::= sum sum ::= product (('+' | '-') product )* product ::= number (('*' | '/') number )* number ::= /[0-9]+/
  41. 41. What next? <ul><li>Variables?
  42. 42. Conditional logic?
  43. 43. For/while loops?
  44. 44. Function calls? </li></ul>
  45. 45. Summary <ul><li>Write the code generator first for instant gratification.
  46. 46. Use Scala parser combinators to build an AST from your input.
  47. 47. Update the code generator to walk the AST, emitting equivalent bytecode for each node using BCEL.
  48. 48. Iterate & add new features! </li></ul>
  49. 49. That's All, Folks! Shine Technologies www http://www.shinetech.com github shinetech twitter @realshinetech Tom Lee www http://tomlee.co email me@tomlee.co github thomaslee twitter @tglee All code from this presentation will be available from http://github.com/thomaslee/oscon2011-awesome

×