The document provides an in-depth look at the Flex compiler and the HellFire Compiler Daemon (HFCD). It discusses the basic architecture of the Flex compiler, including its extensibility, compilation phases, and linker. It also describes the HFCD, a socket-based, out-of-process Flex compiler that can utilize multiple CPUs and machines to speed up builds. A demo of the HFCD is provided, with time for questions and answers at the end.

1. IN-DEPTH LOOK AT THE FLEX
COMPILER AND HFCD
Clement Wong
clement@stopcoding.org
http://stopcoding.wordpress.com

2. AGENDA
• Basic Architecture of Flex Compiler
• Compiler Extensibility
• Overview of Flex Compiler API
• HellFire Compiler Daemon (HFCD)
• Demo, Q/A

3. SPEAKER BIO
• Founder of Bytecode Workshop; Creator of HFCD
• Former Flex Compiler/Profiler Architect & Engineering Lead
• Former ColdFusion MX Architect & Engineering Lead
• Licensed Professional Engineer in the Province of Ontario
• Studied CS and Mech. Eng. at the University of Waterloo

4. FLEX COMPILER TIMELINE

5. FLEX COMPILER 1.0-1.5
• Supports MXML and AS2 (not AS3).
• Based on the AS2 compiler from Flash Authoring.
• Ported from C++ to Java for J2EE.
• Invoked by a MXML servlet.

6. FLEX COMPILER 1.0-1.5
• Major challenges from the AS2 compiler.
• Notarchitected or partitioned for Flex - no clear compilation
phases defined - at least not clear enough for MXML.
• Difficult to extend; not MXML friendly.
• Pre AS3/AVM+: classes and assets were both first-class citizens
- very difficult to implement a good linker.

7. FLEX COMPILER 2.0 - 4.0
• Itis a multi-language compiler. It supports mxml, as, css,
properties and now fxg.
• There is a “mini compiler” for each language. Each “mini
compiler” implements some well-defined compilation phases.
• The “top level” Flex compiler becomes a coordinator -
responsible for looking up source and SWC files; determines
when and which “mini compiler” to call; etc...

8. “MINI COMPILER” EXAMPLES
Package Class
InterfaceCompiler,
MXML flex2.compiler.mxml
ImplementationCompiler
AS3 flex2.compiler.as3 As3Compiler
CSS flex2.compiler.css CssCompiler
ABC flex2.compiler.abc AbcCompiler
Properties flex2.compiler.i18n I18nCompiler

9. SPECIAL CASE: MXML
• Why are there 2 “mini compilers” for MXML?
• InterfaceCompiler
for MXML produces API signature. Does
not produce abc bytecode in generate().
• ImplementationCompiler for MXML generates “full AS3
source code” only after knowing dependent type info.
Generates abc bytecode in generate().
• But why?

10. SPECIAL CASE: MXML
• Consider this: <c:MyComp ...><c:prop .../></c:MyComp>
• Need to know the meaning of every tag in MXML before
accurate MXML-to-AS3 conversion could happen.
• Is “prop” a component or is it a property of “MyComp”?
• We must know the API signature of “MyComp”!

11. COMPILATION PHASES
• preprocess() • analyze4()
• parse1() • generate()
• parse2() • postprocess()
• analyze1()
• analyze2()
• analyze3()

12. COMPILATION PHASES
preprocess() preparation works before parsing.
generates syntax tree; identifies top-level
parse1()
definitions, superclasses and interfaces.
rarely used. used only by the MXML “mini
parse2()
compiler” for data binding.
performs flow analysis. can’t proceed without
analyze1() superclass and interface info. checks syntax tree
and identifies namespaces.

13. COMPILATION PHASES
flow analysis continued. identifies unknown
analyze2()
references.
performs constant and metadata evaluation.
analyze3()
resolve unknown references in syntax trees.
finalizes constant evaluation. all symbols fully
analyze4()
resolved. also performs “lint” evaluation.
performs code generation. produces abc
generate()
bytecode.

14. A SIMPLE ILLUSTRATION
Assume ‘A’ is a subclass of ‘B’. ‘A’ and ‘B’ reference ‘C’.

15. A SIMPLE ILLUSTRATION
Assume ‘A’ is a subclass of ‘B’. ‘A’ and ‘B’ reference ‘C’.

16. A SIMPLE ILLUSTRATION
Assume ‘A’ is a subclass of ‘B’. ‘A’ and ‘B’ reference ‘C’.

17. A SIMPLE ILLUSTRATION
Assume ‘A’ is a subclass of ‘B’. ‘A’ and ‘B’ reference ‘C’.

18. A SIMPLE ILLUSTRATION
Assume ‘A’ is a subclass of ‘B’. ‘A’ and ‘B’ reference ‘C’.

19. A SIMPLE ILLUSTRATION
Assume ‘A’ is a subclass of ‘B’. ‘A’ and ‘B’ reference ‘C’.

20. A SIMPLE ILLUSTRATION
Assume ‘A’ is a subclass of ‘B’. ‘A’ and ‘B’ reference ‘C’.

21. A SIMPLE ILLUSTRATION
Assume ‘A’ is a subclass of ‘B’. ‘A’ and ‘B’ reference ‘C’.

22. A SIMPLE ILLUSTRATION
Assume ‘A’ is a subclass of ‘B’. ‘A’ and ‘B’ reference ‘C’.

23. A SIMPLE ILLUSTRATION
Assume ‘A’ is a subclass of ‘B’. ‘A’ and ‘B’ reference ‘C’.

24. A SIMPLE ILLUSTRATION
Assume ‘A’ is a subclass of ‘B’. ‘A’ and ‘B’ reference ‘C’.

25. A SIMPLE ILLUSTRATION
Assume ‘A’ is a subclass of ‘B’. ‘A’ and ‘B’ reference ‘C’.

26. A SIMPLE ILLUSTRATION
Assume ‘A’ is a subclass of ‘B’. ‘A’ and ‘B’ reference ‘C’.

27. A SIMPLE ILLUSTRATION
Assume ‘A’ is a subclass of ‘B’. ‘A’ and ‘B’ reference ‘C’.

28. A SIMPLE ILLUSTRATION
Assume ‘A’ is a subclass of ‘B’. ‘A’ and ‘B’ reference ‘C’.

29. COMPILER EXTENSIONS
• The “mini
compiler” implementations are “shared” code
among SDK tools.
• mxmlc, compc and asdoc all use the “mini compilers” but they
serve different purposes. They need to do different things at
the end of each compilation phase.
• Solution: “mini compiler” supports “compiler extension”.
• Implement the “flex2.compiler.as3.Extension” interface.

30. EXTENSION EXAMPLES
• ASDocExtension
• DataBindingExtension
• EmbedExtension
• SignatureExtension
• StyleExtension

31. FLEX 4: YAFCEM
• YAFCEM?? Yet Another Flex Compiler Extension Mechanism
• Introducedin Flex 4. Flex developers could insert Java code
into the compiler. The compiler would execute the Java code
at a number of locations defined by the extension interfaces.
• e.g. IApplicationExtension, ILibraryExtension.
• Note: Theyare not executed at the end of the mini compiler
compilation phases. e.g. IApplicationExtension executed at the
end of Application.build().

32. TRANSCODERS
• When there is an [Embed] in your code, how does the
compiler process it?
• Answer: EmbedExtension. EmbedExtension uses transcoders.
• Typically generates an AS3 class and a SWF asset tag.
• The generated classes are handed to the “top-level” compiler
“coordinator”. The classes are then scheduled to be compiled
and linked.

33. TRANSCODERS
• Transcoders implement the “flex2.compiler.Transcoder”
interface.
• The interface looks simple. However, implementation could be
tricky because:
• one must be familiar with the SWF file format.
• one must know what to generate for the asset classes.
• use -keep-generated-actionscript to learn about codegen.

34. TRANSCODER EXAMPLES
• DataTranscoder • SVGTranscoder
• FontTranscoder • morein the
“flex2.compiler.media”
• JPEGTranscoder package.
• LosslessImageTranscoder
• MovieTranscoder
• SoundTranscoder

35. LINKER
• The Flex compiler not just compiles. It also links.
• Application SWFs and library (SWC) SWFs are different.
• Application SWFs typically have 2 frames: Preloader and
essential Flex framework classes in the 1st frame. The rest of
the application and framework classes in the 2nd frame.
• LibrarySWFs are simpler: everything in one frame. Debug info
is always embedded. No SWF optimizations.

36. LINKER
• The SWF structure of these 2 SWF types are encapsulated in
2 Flex classes.
• Application: flex2.linker.FlexMovie
• Library: flex2.compiler.swc.SwcMovie
• Both classes (the generate() method) take a list of compilation
units; run dependency analysis; determine export order and
generate SWF.

37. LINKER
• Theexport order of application SWFs is important because
the ActionScript VM in the Player demands the export order
be based on type dependencies. It is a reasonable demand
because that simplifies VM class loading and verification.
• Use topological sort to determine export order. In fact,
topological sort is used in the compile phase as well (for
processing AS3 inheritance chains in the AS3 compiler and for
circular dependency detection).

38. TOPOLOGICAL SORT
•A vertex represents a definition (e.g. a class or a function).
•A directed edge between two vertices represents the
dependency between the two definitions. For example, when
vertex B points to vertex A, that means A depends on B.
• Ifa vertex has no arrow pointing towards it, it goes to the
export order. The vertex is then removed. Its edges (all
pointing outwards) are also removed. This step continues.

39. TOPOLOGICAL SORT
• The sorting ends when the in-degrees of all the vertices in the
dependency graph are not zero.
• Ifthere are still vertices in the graph, those definitions are in
some circular dependency loops.

40. POST-LINK
• Foreach file compiled, the compiler generates a set of
ActionScript bytecode. It is self-contained, completed with a
constant pool, a set of API signatures and a set of function
bodies.
• Anapplication SWF, if un-optimized, can have a lot of constant
pools. They can be quite overlapping.
• One of the key steps in the post-link phase is “abc merging”.

41. FLEX COMPILER API
• Flex toolchains need a simple way to invoke the compiler.
• Users of the Flex Compiler API:
• Flex Builder
• SAP
• Flex Data Services

42. FLEX COMPILER API
• Recently, one
of the users of the Flex Compiler API is HellFire
Compiler Daemon (HFCD).
• The APIclasses are in the flex2.tools.oem and
flex2.tools.flexbuilder packages.
• Use
flex2.tools.oem.Application to build applications and
modules.
• Use flex2.tools.oem.Library to build libraries.

43. FLEX COMPILER API
Application app = new Application(new File(“HelloWorld.mxml”));
app.setOutput(new File(“HelloWorld.swf ”));
Configuration config = app.getDefaultConfiguration();
SWF config.setXXX(...);
app.setConfiguration(config);
app.build(true);
Library lib = new Library();
lib.addComponent(...);
lib.setOutput(...);
Configuration config = app.getDefaultConfiguration();
SWC config.setSourcePath(...);
config.setXXX(...);
lib.setConfiguration(config);
lib.build(true);

44. FLEX COMPILER API
• The API
usage is simple, but toolchains usually do more... They
implement some of the following compiler interfaces.
• Logger
• ProgressMeter
• PathResolver
• Report

45. HELLFIRE COMPILER DAEMON
• What is HFCD? It is an out-of-process Flex compiler.
• Socket, TCP/IP based. Implemented a RPC version of the Flex
Compiler API.
• Usehardware, multicore technologies to speed up build
performance.
• Eclipse based. Flex Builder compatible. FDT support coming
soon.

46. HFCD ARCHITECTURE

47. HFCD ADVANTAGES
• Socket based. Could run HFCD on a second machine (i.e.
access to more CPU and memory resources).
• Use Java server VM to run the compiler. Consistently 30%
faster than Java client VM.
• Withmultiple CPU cores, HFCD builds applications/libraries
concurrently if possible. Build speedup factor could reach the
number of available CPUs.

48. HFCD ADVANTAGES
• Use
HFCD ant tasks, the ant <parallel> task and multiple
HFCD servers to build.
• For more information, please visit:
• http://bytecode-workshop.com
• http://stopcoding.wordpress.com

49. HFCD DEMO

50. Q/A