Motivation Automatically generate serialization/de- serialization code for any give primitive or composite data type. Directly plug serialization/de-serialization code to generate MapReduce output file that supports RC storage format. An extension to the existing Hadoop Record Compiler (Jute) package – So is named JuteRc.
Haoop Record Compiler Two important packages: Hadoop.record.compiler.generated – rcc.jj Hadoop.record.compiler CodeGenerator JType : Jboolean, Jstring, JCompType (JMap, Jvector..) Execution path: bin/rcc -> rcc#JFile.genCode -> CodeGenerator#genCode
Haoop Record Compiler cont. Inside CondeGenerator JRecord .codeGen : set up members, set up member functions: setter/getter, serialization/de-serialization (JType.getRead/WriteMethod) abastract JType (JavaType, CppType) JBoolean, JString, JMap, Jvector JType.getRead/WriteMethod ( important)
Haoop Record Compiler cont. JType.getReadMethod _rio_a.startRecord(this,_rio_tag); _rio_a.writeBuffer(bcookie,"bcookie"); _rio_a.writeLong(timestamp,"timestamp"); _rio_a.endRecord(this,_rio_tag);