Intel JIT Talk

1.
High Performance JavaScript:JITs in Firefox

2.
Power of theWeb

3.
How did weget here?Making existing web tech faster and faster

4.
Adding new technologies,like video, audio, integration and 2D+3D drawingThe Browser

5.
Getting a Webpagewww.intel.com192.198.164.158

6.
Getting a Webpagewww.intel.com192.198.164.158GET/ HTTP/1.0(HTTP Request)

7.
Getting a Webpage<html><head><title>Laptop,Notebooks, ...</title></head><script language="javascript">functiondoStuff() {...}</script><body>...(HTML, CSS, JavaScript)

8.
Core Web ComponentsHTMLprovides content

9.
CSS provides styling

10.
JavaScript provides aprogramming interfaceNew Technologies2D Canvas

11.
Video, Audio tags

12.
WebGL (OpenGL +JavaScript)DOM(Source: http://www.w3schools.com/htmldom/default.asp)

13.
CSSCascading Style SheetsSeparatespresentation from content<b class=”warning”>This is red</b>.warning {color: red;text-decoration: underline;}

14.
What is JavaScript?C-likesyntax

15.
De facto languageof the web

16.
Interacts with theDOM and the browserC-Like SyntaxBraces, semicolons, familiar keywordsif (x) {for (i=0; i<100; i++)print("Hello!");}

17.
Displaying a WebpageLayoutengine applies CSS to DOMComputes geometry of elementsJavaScript engine executes codeThis can change DOM, triggers “reflow”Finished layout is sent to graphics layer

18.
The Result

19.
JavaScriptInvented by BrendanEich in 1995 for Netscape 2

20.
Initial version writtenin 10 days

21.
Anything from smallbrowser interactions, complex apps (Gmail) to intense graphics (WebGL)The Result

22.
Think LISP orSelf, not JavaUntyped - no type declarationsMulti-Paradigm – objects, closures, first-class functionsHighly dynamic - objects are dictionaries

23.
No Type DeclarationsProperties,variables, return values can be anything:functionf(a) {var x =72.3;if (a) x = a +"string";return x;}

24.
No Type DeclarationsProperties,variables, return values can be anything:if a is: number: add; string: concat;functionf(a) {var x =“hi”;if (a) x = a +33.2;return x;}

25.
FunctionalFunctions may bereturned, passed as arguments:functionf(a) {returnfunction () {return a; }}var m = f(5);print(m());

26.
ObjectsObjects are dictionariesmapping strings to values

27.
Properties may bedeleted or added at any time!var point = { x : 5, y : 10 };deletepoint.x;point.z=12;

28.
PrototypesEvery object canhave a prototype objectIf a property is not found on an object, its prototype is searched instead… And the prototype’s prototype, etc..

29.
NumbersJavaScript specifies thatnumbers are IEEE-754 64-bit floating point

30.
Engines use 32-bitintegers to optimize

31.
Must preserve semantics:integers overflow to doublesNumbersvarx = 0x7FFFFFFF;x++;intx = 0x7FFFFFFF;x++;

32.
JavaScript ImplementationsValues (Objects,strings, numbers)

33.
Garbage collector

34.
Runtime library

35.
Execution engine (VM)ValuesTheruntime must be able to query a value’s type to perform the right computation.When storing values to variables or object fields, the type must be stored as well.

36.
ValuesBoxed values requiredfor variables, object fields

37.
Unboxed values requiredfor computationBoxed ValuesNeed a representation in C++

38.
Easy idea: 96-bitstruct

39.
32-bits for type

40.
64-bits for double,pointer, or integer

41.
Too big! Wehave to pack better.Boxed ValuesWe could use pointers, and tag the low bitsDoubles would have to be allocated on the heapIndirection, GC pressure is badThere is a middle ground…

42.
Values are 64-bit

43.
Doubles are normalIEEE-754

44.
How to packnon-doubles?

45.
51 bits ofNaN space!IA32, ARMNunboxing

46.
NunboxingIA32, ARMTypePayload0x400c00000x00000000630Full value:0x400c000000000000

47.
Type is doublebecause it’s not a NaN

48.
Encodes: (Double, 3.5)NunboxingIA32,ARMTypePayload0x000000400xFFFF0001630Full value: 0xFFFF000100000040

49.
Value is inNaN space

50.
Type is 0xFFFF0001(Int32)

51.
Value is (Int32,0x00000040)Punboxingx86-64TypePayload630471111 1111 1111 1000 0000 .. 0000 0000 0000 0000 0100 0000Full value: 0xFFFF800000000040

52.
Value is inNaN space

53.
Bits >> 47== 0x1FFFF == INT32

54.
Bits 47-63 maskedoff to retrieve payload

55.
Value(Int32, 0x00000040)* Some64-bit OSes can restrict mmap() to 32-bits

56.
Garbage CollectionNeed toreclaim memory without pausing user workflow, animations, etcNeed very fast object allocationConsider lots of small objects like points, vectorsReading and writing to the heap must be fast

57.
Mark and SweepAlllive objects are found via a recursive traversal of the root value setAll dead objects are added to a free listVery slow to traverse entire heapBuilding free lists can bring “dead” memory into cache

58.
ImprovementsSweeping, freelist buildinghave been moved off-thread

59.
Marking can beincremental, interleaved with program executionGenerational GCObservation: most objects are short-livedAll new objects are bump-allocated into a newborn spaceOnce newborn space is full, live objects are moved to the tenured spaceNewborn space is then reset to empty

60.
Generational GCInactiveActiveNewborn Space8MB8MBTenuredSpace

61.
After Minor GCActiveInactiveNewbornSpace8MB8MBTenured Space

62.
BarriersIncremental marking, generationalGC need read or write barriersRead barriers are much slowerAzul Systems has hardware read barrierWrite barrier seems to be well-predicted?

63.
UnknownsHow much docache effects matter?Memory GC has to touchLocality of objectsWhat do we need to consider to make GC fast?

64.
Running JavaScriptInterpreter –Runs code, not fast

65.
Basic JIT –Simple, untyped compiler

66.
Advanced JIT –Typed compiler, lightspeedInterpreterGood for code that runs once

67.
Giant switch loop

68.
Handles all edgecases of JS semanticsInterpreterwhile (true) {switch (*pc) {case OP_ADD: ...case OP_SUB: ...case OP_RETURN: ... } pc++;}

69.
Interpretercase OP_ADD: { Value lhs = POP(); Value rhs = POP(); Value result;if (lhs.isInt32() && rhs.isInt32()) {int left = rhs.toInt32();int right = rhs.toInt32();if (AddOverflows(left, right, left + right)) result.setInt32(left + right);elseresult.setNumber(double(left) + double(right)); } elseif (lhs.isString() || rhs.isString()) { String *left = ValueToString(lhs); String *right = ValueToString(rhs); String *r = Concatenate(left, right);result.setString(r); } else {double left = ValueToNumber(lhs);double right = ValueToNumber(rhs);result.setDouble(left + right); }PUSH(result);break;

70.
InterpreterVery slow!Lots ofopcode and type dispatchLots of interpreter stack trafficJust-in-time compilation solves both

71.
Just In TimeCompilationmust be very fastCan’t introduce noticeable pausesPeople care about memory useCan’t JIT everythingCan’t create bloated codeMay have to discard code at any time

72.
Basic JITJägerMonkey inFirefox 4Every opcode has a hand-coded template of assembly (registers left blank)Method at a time:Single pass through bytecode stream!Compiler uses assembly templates corresponding to each opcode

73.
BytecodeGETARG 0 ;fetch xGETARG 1 ; fetch yADDRETURNfunctionAdd(x, y) {return x + y;}

74.
Interpretercase OP_ADD: { Value lhs = POP(); Value rhs = POP(); Value result;if (lhs.isInt32() && rhs.isInt32()) {int left = rhs.toInt32();int right = rhs.toInt32();if (AddOverflows(left, right, left + right)) result.setInt32(left + right);elseresult.setNumber(double(left) + double(right)); } elseif (lhs.isString() || rhs.isString()) { String *left = ValueToString(lhs); String *right = ValueToString(rhs); String *r = Concatenate(left, right);result.setString(r); } else {double left = ValueToNumber(lhs);double right = ValueToNumber(rhs);result.setDouble(left + right); }PUSH(result);break;

75.
Assembling ADDInlining thathuge chunk for every ADD would be very slow

76.
Observation:

77.
Some input typesmuch more common than othersInteger Math – Commonvar j =0;for (i=0; i<10000; i++) { j +=i;}Weird Stuff – Rare!var j =12.3;for (i =0; i <10000; i++) { j +=new Object() + i.toString();}

78.
Assembling ADDOnly generatecode for the easiest and most common cases

79.
Large design space

80.
Can consider integerscommon, or

81.
Integers and doubles,or

82.
Anything!Basic JIT -ADDif (arg0.type != INT32) gotoslow_add;if (arg1.type != INT32)gotoslow_add;

83.
Basic JIT -ADDif (arg0.type != INT32) gotoslow_add;if (arg1.type != INT32)gotoslow_add;R0 = arg0.dataR1 = arg1.dataGreedy Register Allocator

84.
Basic JIT -ADDif (arg0.type != INT32) gotoslow_add;if (arg1.type != INT32)gotoslow_add;R0 = arg0.data;R1 = arg1.data;R2 = R0 + R1;if (OVERFLOWED)gotoslow_add;

85.
Slow Pathsslow_add: Value result = runtime::Add(arg0, arg1);

86.
InlineOut-of-lineif (arg0.type !=INT32) gotoslow_add;if (arg1.type != INT32)gotoslow_add;R0 = arg0.data;R1 = arg1.data;R2 = R0 + R1; if (OVERFLOWED)gotoslow_add;rejoin:slow_add: Value result = Interpreter::Add(arg0, arg1);R2 = result.data;goto rejoin;

87.
RejoiningInlineOut-of-line if(arg0.type != INT32) gotoslow_add; if (arg1.type != INT32)gotoslow_add; R0 = arg0.data; R1 = arg1.data; R2 = R0 + R1; if (OVERFLOWED)gotoslow_add;R3 = TYPE_INT32;rejoin:slow_add: Value result = runtime::Add(arg0, arg1); R2 = result.data; R3 = result.type;goto rejoin;

88.
Final CodeInlineOut-of-lineif (arg0.type!= INT32) goto slow_add; if (arg1.type != INT32)goto slow_add;R0 = arg0.data;R1 = arg1.data;R2 = R0 + R1; if (OVERFLOWED)goto slow_add;R3 = TYPE_INT32;rejoin:return Value(R3, R2);slow_add: Value result = Interpreter::Add(arg0, arg1);R2 = result.data;R3 = result.type;goto rejoin;

89.
ObservationsEven though wehave fast paths, no type information can flow in between opcodesCannot assume result of ADD is integerMeans more branchesTypes increase register pressure

90.
Basic JIT –Object Accessx.yis multiple dictionary searches!

91.
How can wecreate a fast-path?

92.
We could tryto cache the lookup, but

93.
We want itto work for >1 objectsfunctionf(x) {returnx.y;}

94.
Object AccessObservationUsually, objectsflowing through an access site look the sameIf we knew an object’s layout, we could bypass the dictionary lookup

95.
Object Layoutvar obj= { x: 50, y: 100, z: 20 };

96.
Object Layout

97.
Object Layoutvarobj ={ x: 50, y: 100, z: 20 };…varobj2 = { x: 78, y: 93, z: 600 };

98.
Object Layout

99.
Familiar ProblemsWe don’tknow an object’s shape during JIT compilation... Or even that a property access receives an object!Solution: leave code “blank,” lazily generate it later

100.
Inline Cachesfunctionf(x) {returnx.y;}

101.
Inline CachesStart asnormal, guarding on type and loading dataif (arg0.type != OBJECT)gotoslow_property;JSObject *obj = arg0.data;

102.
No information yet:leave blank (nops). if (arg0.type != OBJECT)gotoslow_property;JSObject *obj = arg0.data;gotoproperty_ic;Inline Caches

103.
Property ICsfunctionf(x) {returnx.y;}f({x:5, y: 10, z: 30});

104.
Property ICs

105.
Property ICs

106.
Property ICs

107.
Inline CachesShape =S1, slot is 1 if (arg0.type != OBJECT)gotoslow_property;JSObject *obj = arg0.data;gotoproperty_ic;

108.
Inline CachesShape =S1, slot is 1 if (arg0.type != OBJECT)gotoslow_property;JSObject *obj = arg0.data;gotoproperty_ic;

109.
Inline CachesShape =SHAPE1, slot is 1 if (arg0.type != OBJECT)gotoslow_property;JSObject *obj = arg0.data; if (obj->shape != SHAPE1)gotoproperty_ic;R0 = obj->slots[1].type;R1 = obj->slots[1].data;

110.
PolymorphismWhat happens iftwo different shapes pass through a property access?

111.
Add another fast-path...functionf(x){returnx.y;}f({ x: 5, y: 10, z: 30});f({ y: 40});

112.
PolymorphismMain Method if(arg0.type != OBJECT)gotoslow_path;JSObject *obj = arg0.data;if (obj->shape != SHAPE1)gotoproperty_ic;R0 = obj->slots[1].type;R1 = obj->slots[1].data;rejoin:

113.
PolymorphismMain MethodGenerated Stub if (arg0.type != OBJECT)gotoslow_path;JSObject *obj = arg0.data; if (obj->shape != SHAPE0)gotoproperty_ic; R0 = obj->slots[1].type; R1 = obj->slots[1].data;rejoin: stub:if (obj->shape != SHAPE2)gotoproperty_ic;R0 = obj->slots[0].type;R1 = obj->slots[0].data;goto rejoin;

114.
PolymorphismMain MethodGenerated Stubif(arg0.type != OBJECT) goto slow_path; JSObject *obj = arg0.data;if (obj->shape != SHAPE1)goto stub; R0 = obj->slots[1].type; R1 = obj->slots[1].data;rejoin: stub:if (obj->shape != SHAPE2)goto property_ic;R0 = obj->slots[0].type;R1 = obj->slots[0].data;goto rejoin;

115.
Chain of Stubsif(obj->shape== S1) { result = obj->slots[0];} else {if (obj->shape == S2) { result = obj->slots[1]; } else {if (obj->shape == S3) { result = obj->slots[2]; } else {if (obj->shape == S4) { result = obj->slots[3]; } else {goto property_ic; } } }}

116.
Code MemoryGenerated codeis patched from inside the method

117.
Self-modifying, but singlethreaded

118.
Code memory isalways rwx

119.
Concerned about protection-flippingexpenseBasic JIT SummaryGenerates simple code to handle common cases

120.
Inline caches adaptcode based on runtime observations

121.
Lots of guards,poor type informationOptimizing HarderSingle pass too limited: we want to perform whole-method optimizations

122.
We could generatean IR, but without type information...

123.
Slow paths preventmost optimizations.Code Motion?functionAdd(x, n) {varsum = 0;for (vari = 0; i < n; i++) sum += x + n; return sum;}The addition looks loop invariant.Can we hoist it?

124.
Code Motion?functionAdd(x, n){varsum = 0;vartemp0 = x + n;for (vari = 0; i < n; i++) sum += temp0; return sum;}Let’s try it.

125.
Wrong Resultvar global=0;varobj= {valueOf: function () {return++global; }}Add(obj, 10);Original code returns 155

126.
Hoisted version returns110!IdempotencyUntyped operations are usually not provably idempotentSlow paths are re-entrant, can have observable side effectsThis prevents code motion, redundancy elimination

127.
Ideal ScenarioActual knowledgeabout types!

128.
Remove slow paths

129.
Perform whole-method optimizationsAdvancedJITsMake optimistic guesses about types

130.
Guesses must beinformed

131.
Don’t want towaste time compiling code that can’t or won’t run

132.
Using type inferenceor runtime profiling

133.
Generate an IR,perform textbook compiler optimizationsOptimism Pays OffPeople naturally write code as if it were typedVariables and object fields that change types are rare

134.
IonMonkeyWork in progress

135.
Constructs high andlow-level IRs

136.
Applies type informationusing runtime feedback

137.
Inlining, GVN, LICM,LSRAGETARG 0 ; fetch xGETARG 1 ; fetch yADDGETARG 0 ; fetch xADDRETURNfunctionAdd(x, y) {return x + y + x;}

138.
Build SSAGETARG 0; fetch xGETARG 1 ; fetch yADDGETARG 0 ; fetch xADDRETURNv0 = arg0v1 = arg1

139.
Build SSAGETARG 0; fetch xGETARG 1 ; fetch yADDGETARG 0 ; fetch xADDRETURNv0 = arg0v1 = arg1

140.
Type OracleNeed amechanism to inform compiler about likely typesType Oracle: given program counter, returns types for inputs and outputsMay use any data source – runtime profiling, type inference, etc

141.
Type OracleFor thisexample, assume the type oracle returns “integer” for the output and both inputs to all ADD operations

142.
Build SSAGETARG 0; fetch xGETARG 1 ; fetch yADDGETARG 0 ; fetch xADDRETURNv0 = arg0v1 = arg1v2 = iadd(v0, v1)

143.
Build SSAGETARG 0; fetch xGETARG 1 ; fetch yADDGETARG 0 ; fetch xADDRETURNv0 = arg0v1 = arg1v2 = iadd(v0, v1)v3 = iadd(v2, v0)

144.
Build SSAGETARG 0; fetch xGETARG 1 ; fetch yADDGETARG 0 ; fetch xADDRETURNv0 = arg0v1 = arg1v2 = iadd(v0, v1)v3 = iadd(v2, v0)v4 = return(v3)

145.
SSA (Intermediate)v0 =arg0v1 = arg1v2 = iadd(v0, v1)v3 = iadd(v2, v0)v4 = return(v3)UntypedTyped

146.
Intermediate SSASSA doesnot type check

147.
Type analysis:

148.
Makes sure SSAinputs have correct type

149.
Inserts conversions, valuedecodingUnoptimized SSAv0 = arg0v1 = arg1v2 = unbox(v0, INT32)v3 = unbox(v1, INT32)v4 = iadd(v2, v3)v5 = unbox(v0, INT32)v6 = iadd(v4, v5)v7 = return(v6)

150.
Optimizationv0 = arg0v1= arg1v2 = unbox(v0, INT32)v3 = unbox(v1, INT32)v4 = iadd(v2, v3)v5 = unbox(v0, INT32)v6 = iadd(v4, v5)v7 = return(v6)

151.
Optimizationv0 = arg0v1= arg1v2 = unbox(v0, INT32)v3 = unbox(v1, INT32)v4 = iadd(v2, v3)v5 = unbox(v0, INT32)v6 = iadd(v4, v5)v7 = return(v6)

152.
Optimized SSAv0 =arg0v1 = arg1v2 = unbox(v0, INT32)v3 = unbox(v1, INT32)v4 = iadd(v2, v3)v5 = iadd(v4, v2)v6 = return(v5)

153.
Register AllocationLinear ScanRegister AllocationBased on Christian Wimmer’s workLinear Scan Register Allocation for the Java HotSpot™ Client Compiler. Master's thesis, Institute for System Software, Johannes Kepler University Linz, 2004Linear Scan Register Allocation on SSA Form. In Proceedings of the International Symposium on Code Generation and Optimization, pages 170–179. ACM Press, 2010.Same algorithm used in Hotspot JVM

154.
Allocate Registers0: stackarg01: stack arg12: r1 unbox(0, INT32)3: r0 unbox(1, INT32)4: r0 iadd(2, 3)5: r0 iadd(4, 2)6: r0 return(5)

155.
Code GenerationSSA0: stackarg01: stack arg12: r1 unbox(0, INT32)3: r0 unbox(1, INT32)4: r0 iadd(2, 3)5: r0 iadd(4, 2)6: r0 return(5)

156.
Code GenerationSSANative Code0:stack arg01: stack arg12: r1unbox(0, INT32)3: r0 unbox(1, INT32)4: r0 iadd(2, 3)5: r0 iadd(4, 2)6: r0 return(5)if (arg0.type != INT32)goto BAILOUT; r1 = arg0.data;

157.
Code GenerationSSANative Code0:stack arg01: stack arg12: r1 unbox(0, INT32)3: r0unbox(1, INT32)4: r0 iadd(2, 3)5: r0 iadd(4, 2)6: r0 return(5) if (arg0.type != INT32) goto BAILOUT; r1 = arg0.data;if (arg1.type != INT32)goto BAILOUT; r0 = arg1.data;

158.
Code GenerationSSANative Code0:stack arg01: stack arg12: r0 unbox(0, INT32)3: r1 unbox(1, INT32)4: r0iadd(2, 3)5: r0 iadd(4, 2)6: r0 return(5) if (arg0.type != INT32) goto BAILOUT; r0 = arg0.data; if (arg1.type != INT32) goto BAILOUT; r1 = arg1.data; r0 = r0 + r1;if (OVERFLOWED)goto BAILOUT;

159.
Code GenerationSSANative Code0:stack arg01: stack arg12: r0 unbox(0, INT32)3: r1 unbox(1, INT32)4: r0 iadd(2, 3)5: r0iadd(4, 2)6: r0 return(5) if (arg0.type != INT32) goto BAILOUT; r0 = arg0.data; if (arg1.type != INT32) goto BAILOUT; r1 = arg1.data; r0 = r0 + r1; if (OVERFLOWED) goto BAILOUT; r0 = r0 + r1;if (OVERFLOWED)goto BAILOUT;

160.
Code GenerationSSANative Code0:stack arg01: stack arg12: r0 unbox(0, INT32)3: r1 unbox(1, INT32)4: r0 iadd(2, 3)5: r0 iadd(4, 2)6: r0 return(4) if (arg0.type != INT32) goto BAILOUT; r0 = arg0.data; if (arg1.type != INT32) goto BAILOUT; r1 = arg1.data; r0 = r0 + r1; if (OVERFLOWED) goto BAILOUT; r0 = r0 + r1; if (OVERFLOWED) goto BAILOUT; return r0;

161.
Generated Code if(arg0.type != INT32)goto BAILOUT; r0 = arg0.data; if (arg1.type != INT32)goto BAILOUT; r1 = arg1.data; r0 = r0 + r1; if (OVERFLOWED)goto BAILOUT; r0 = r0 + r1; if (OVERFLOWED)goto BAILOUT;return r0;

162.
Generated CodeBailout exitsJIT

163.
May recompile functionif (arg0.type != INT32)goto BAILOUT; r0 = arg0.data; if (arg1.type != INT32)goto BAILOUT; r1 = arg1.data; r0 = r0 + r1; if (OVERFLOWED)goto BAILOUT; r0 = r0 + r1; if (OVERFLOWED)goto BAILOUT;return r0;

164.
Guards Still NeededObjectshapes for reading propertiesTypes of…ArgumentsValues read from the heapValues returned from C++

165.
Guards Still NeededMathIntegerOverflowDivide by ZeroMultiply by -0NaN is falseyin JS, truthy in ISAs

166.
Advanced JITsFull typespeculation - no slow paths

167.
Can move andeliminate code

168.
If speculation fails,method is recompiled or deoptimized

169.
Can still usetechniques like inline caching!To Infinity, and BeyondAdvanced JIT example has four guards:

170.
Two type checks

171.
Two overflow checks

172.
Better than BasicJIT, but we can do better

173.
Interval analysis canremove overflow checks

174.
Type Inference!Type InferenceWholeprogram analysis to determine types of variables

175.
Hybrid: both staticand dynamic

176.
Replaces checks inJIT with checks in virtual machine

177.
If VM breaksan assumption held in any JIT code, that JIT code is discardedConclusionsThe web needs JavaScript to be fastUntyped, highly dynamic nature makes this challengingValue boxing has different tradeoffs on different ISAsGC needs to be fast – what should we focus on?

178.
ConclusionsJITs getting morepowerfulType specialization is keyStill need checks and special cases on many operations

179.
Questions?

Intel JIT Talk

More Related Content

What's hot

Similar to Intel JIT Talk

Recently uploaded

Intel JIT Talk

Editor's Notes