Common wisdom dictates that native languages are the only means of building high-performance applications. How do managed runtimes such as those available to .NET, Java, and even JavaScript, yes even JavaScript compare? Many applications requiring high-performance are now developed for managed runtimes - such as financial trading, data stores and analytics, messaging processing, and even supercomputing. Over the last few decades we have seen significant advances in managed runtimes, particularly for JIT compilers and garbage collectors. In this talk we will explore how our managed runtimes can equal, and even better in some cases, the performance of native languages.

1. High Performance
Managed Languages
Martin Thompson - @mjpt777

2. Really, what is your preferred
platform for building HFT
applications?

4. Why would you build
low-latency applications on a
GC’ed platform?

6. 1. Some Context
2. Runtime Optimisation
3. Garbage Collection
4. Algorithms & Design
Agenda

7. Some Context

8. Let’s be clear
A Managed Runtime is not
always a good choice…

9. Latency Arbitrage?

11. Are native languages faster?

13. Time?
Skills & Resources?

14. CPU vs Memory
Performance

15. Time to add 2 integers?

16. 1 CPU Cycle : < 1ns
http://www.agner.org/optimize/instruction_tables.pdf

17. Time per operation to sum the
values in an array of integers?

18. Access Pattern Benchmark
Benchmark Score Error Units
===========================================
sequential 0.832 ± 0.006 ns/op

19. Really???
Less than 1ns per operation?

21. What if the access pattern
is different?

22. Access Pattern Benchmark
Benchmark Score Error Units
===========================================
sequential 0.832 ± 0.006 ns/op
randomPage 2.703 ± 0.025 ns/op
dependentRandomPage 7.102 ± 0.326 ns/op
randomHeap 19.896 ± 3.110 ns/op
dependentRandomHeap 89.516 ± 4.573 ns/op

23. Data Dependent Loads
aka “Pointer Chasing”!!!

24. Performance 101

25. 1. Memory is transported in Cachelines
Performance 101

26. 1. Memory is transported in Cachelines
2. Memory is managed in OS Pages
Performance 101

27. 1. Memory is transported in Cachelines
2. Memory is managed in OS Pages
3. Memory is pre-fetched on
predictable access patterns
Performance 101

28. Runtime Optimisation

29. 1. Profile guided optimisations
Runtime JIT

30. 1. Profile guided optimisations
2. Bets can be taken and later revoked
Runtime JIT

31. Branches
void foo()
{
// code
if (condition)
{
// code
}
// code
}

32. Block A
Branches
void foo()
{
// code
if (condition)
{
// code
}
// code
}

33. Block A
Block B
Branches
void foo()
{
// code
if (condition)
{
// code
}
// code
}

34. Block A
Block C
Block B
Branches
void foo()
{
// code
if (condition)
{
// code
}
// code
}

35. Block A
Block C
Block B
Branches
void foo()
{
// code
if (condition)
{
// code
}
// code
}
Block A
Block C

36. Block A
Block C
Block B
Branches
void foo()
{
// code
if (condition)
{
// code
}
// code
}
Block A
Block C
Block B

37. Subtle Branches
int result = (i > 7) ? a : b;

38. Subtle Branches
int result = (i > 7) ? a : b;
CMOV vs Branch Prediction?

39. Method/Function Inlining
void foo()
{
// code
bar();
// code
}

40. Method/Function Inlining
void foo()
{
// code
bar();
// code
}
Block A

41. Method/Function Inlining
void foo()
{
// code
bar();
// code
}
Block A
bar()

42. Method/Function Inlining
void foo()
{
// code
bar();
// code
}
Block A
bar()

43. Method/Function Inlining
void foo()
{
// code
bar();
// code
}
Block A
Block B
bar()

44. Method/Function Inlining
void foo()
{
// code
bar();
// code
}
Block A
Block B
bar()
Block A

45. Method/Function Inlining
void foo()
{
// code
bar();
// code
}
Block A
Block B
bar()
Block A
bar()

46. Method/Function Inlining
void foo()
{
// code
bar();
// code
}
Block A
Block B
bar()
Block A
Block B
bar()

47. Method/Function Inlining
void foo()
{
// code
bar();
// code
}
i-cache
& code bloat?

48. “Inlining is THE optimisation.”
- Cliff Click
Method/Function Inlining

49. void foo(int[] array, int length)
{
for (int i = 0; i < length; i++)
{
bar(Integer.bitCount(array[i]));
}
}
Loops

50. void foo(int[] array, int length)
{
for (int i = 0; i < length; i += 4)
{
bar(Integer.bitCount(array[i]));
bar(Integer.bitCount(array[i + 1]));
bar(Integer.bitCount(array[i + 2]));
bar(Integer.bitCount(array[i + 3]));
}
}
Loops

51. Loops
void foo(int[] array, int length)
{
for (int i = 0; i < length; i++)
{
bar(Integer.bitCount(array[i]));
}
}
Intrinsics

52. Subtype Polymorphism
void draw(Shape[] shapes)
{
for (int i = 0; i < shapes.length; i++)
{
shapes[i].draw();
}
}

53. Subtype Polymorphism
void draw(Shape[] shapes)
{
for (int i = 0; i < shapes.length; i++)
{
shapes[i].draw();
}
}
void bar(Shape shape)
{
bar(shape.isVisible());
}

54. Subtype Polymorphism
void draw(Shape[] shapes)
{
for (int i = 0; i < shapes.length; i++)
{
shapes[i].draw();
}
}
void bar(Shape shape)
{
bar(shape.isVisible());
}
Class Hierarchy Analysis
& Inline Caching

55. 1. Profile guided optimisations
2. Bets can be taken and later revoked
Runtime JIT

56. Garbage Collection

57. Generational Garbage Collection
“Only the good die young”
- Billy Joel

58. Eden Survivor 0 Survivor 1
Young/New Generation
TLAB
TLAB
Tenured
Virtual
Virtual
Old Generation
Generational Garbage Collection

59. Modern Hardware (Intel Sandy Bridge EP)
C 1 C n C 1 C nRegisters/Buffers <1ns
L1 L1 L1 L1~4 cycles ~1ns
L2 L2 L2 L2~12 cycles ~3ns
L3 L3
~40 cycles ~15ns
~75 cycles ~25ns (dirty hit)
~65ns
DRAM
QPI ~40ns
MC MC
DRAM
DRAM
DRAM
DRAM
DRAM
DRAM
DRAM
...
...
...
...
...
...
QPI QPIPCI-e 3 PCI-e 3
40X
IO
40X
IO
* Assumption: 3GHz Processor

60. Broadwell EX – 24 cores & 60MB L3 Cache

61. Eden
Young/New Generation
TLAB
TLAB
Thread Local Allocation Buffers

62. Eden
Young/New Generation
TLAB
TLAB
Thread Local Allocation Buffers
• Affords locality of reference
• Avoid false sharing
• Can have NUMA aware allocation

63. Eden Survivor 0 Survivor 1
Young/New Generation
TLAB
TLAB
Virtual
Object Survival

64. Eden Survivor 0 Survivor 1
Young/New Generation
TLAB
TLAB
Virtual
Object Survival
• Aging Policies
• Compacting Copy
• NUMA Interleave
• Fast Parallel Scavenging
• Only the survivors require work

65. Eden Survivor 0 Survivor 1
Young/New Generation
TLAB
TLAB
Tenured
Virtual
Virtual
Old Generation
Object Promotion

66. Eden Survivor 0 Survivor 1
Young/New Generation
TLAB
TLAB
Tenured
Virtual
Virtual
Old Generation
Object Promotion
• Concurrent Collection
• String Deduplication

67. Compacting Collections

68. Compacting Collections – Depth first copy

69. Compacting Collections

70. Compacting Collections
OS Pages and
cache lines?

71. EdenE
O
S
O
S
O E
S OO
E
O E O O
O
Survivor
Old
Unused
O
S
E
H
HumongousH
H
O O
G1 – Concurrent Compaction?

72. Azul Zing C4
True Concurrent Compacting
Collector

73. Where next for GC?

74. Object Inlining/Aggregation

75. GC vs Manual Memory Management
Not easy to pick clear winner…

76. GC vs Manual Memory Management
Managed GC
• GC Implementation
• Card Marking
• Read/Write Barriers
• Object Headers
• Background Overhead
on CPU and Memory
Not easy to pick clear winner…

77. GC vs Manual Memory Management
Managed GC
• GC Implementation
• Card Marking
• Read/Write Barriers
• Object Headers
• Background Overhead
on CPU and Memory
Not easy to pick clear winner…
Native
• Malloc Implementation
• Arena/pool contention
• Bin Wastage
• Fragmentation
• Debugging Effort
• Inter-thread costs

78. Algorithms & Design

79. What is most important to
performance?

80. Time

81. “If I had more time, I would
have written a shorter letter.”
- Blaise Pascal

82. • Avoiding duplicate work
• Avoiding cache misses
• Avoiding contention
• Strength reduction
• Amortising expensive operations
• Mechanical Sympathy
• Choice of algorithms & data structures
• API design

83. In a large codebase it is really
difficult to do everything well

84. It also takes some “uncommon”
disciplines such as:
profiling, telemetry, modelling…

85. The story of Aeron
https://github.com/real-logic/Aeron

86. Aeron is an interesting lesson in
“time to performance”

87. Lots of others exists such at the
C# Roslyn compiler

88. Time spent on
Mechanical Sympathy
vs
Debugging Pointers
???

89. GC
Immutable Data & Concurrency

90. In Closing …

91. What does the future hold?

92. Remember
Assembly vs Compiled
Languages?

93. What about
footprint, startup, warm up, etc.
???

97. Blog: http://mechanical-sympathy.blogspot.com/
Twitter: @mjpt777
“Any intelligent fool can make things bigger, more
complex, and more violent.
It takes a touch of genius, and a lot of courage, to move
in the opposite direction.”
- Albert Einstein
Questions?