The document discusses the development of Aeron, a new messaging protocol and library designed to address issues of feature bloat, complexity, and performance in modern multi-core and cloud environments. Key design principles include non-blocking I/O, garbage-free operations, and efficient message path handling, allowing for high throughput and low latency messaging. It also touches on the evolution of message structures and protocols, as well as future directions for the project, including enhancements in replication and performance.

1. Use$promo$code!“ilovegoto”!!
for$$100$off$registration!

2. Aeron
What, Why, and What Next?
Todd L. Montgomery
@toddlmontgomery

3. THANK YOU!

5. 1. Why build another Product?
2. What Features are really needed?
3. How does one Design for this?
4. How did things Evolve along the way?
5. What’s Next?

7. 1. Why build another
product?

8. Feature Bloat & Complexity

9. Not Fast Enough

10. Low & Predictable Latency is key

11. We are in a new world
Multi-core, Multi-socket,
Cloud...

12. We are in a new world
UDP, IPC, InfiniBand,
RDMA, PCI-e
Multi-core, Multi-socket,
Cloud...

13. Aeron is trying a new approach

14. The Team
Todd Montgomery Richard Warburton
Martin Thompson

15. 2. What features
are really needed?

16. Publishers SubscribersChannel
Stream
Messaging
Channel

17. A library, not a framework, on
which other abstractions and
applications can be built

18. Composable Design

19. OSI layer 4 Transport for
message oriented streams

20. OSI Layer 4 (Transport) Services
1. Connection Oriented Communication
2. Reliability
3. Flow Control
4. Congestion Avoidance/Control
5. Multiplexing

21. Connection Oriented Communication

22. Reliability

23. Flow Control

24. Congestion Avoidance/Control

25. Multiplexing

26. Multi-Everything World!

27. Publishers Subscribers
Channel
Stream
Multi-Everything World

28. 3. How does one
design for this?

29. Design Principles
1. Garbage free in steady state running
2. Smart Batching in the message path
3. Wait-free algos in the message path
4. Non-blocking IO in the message path
5. No exceptional cases in message path
6. Apply the Single Writer Principle
7. Prefer unshared state
8. Avoid unnecessary data copies

30. It’s all about 3 things

31. It’s all about 3 things
1. System Architecture

32. It’s all about 3 things
1. System Architecture
2. Data Structures

33. It’s all about 3 things
1. System Architecture
2. Data Structures
3. Protocols of Interaction

34. Publisher
Subscriber
Subscriber
Publisher
Architecture
IPC Log Buffer

35. Sender
Receiver
Receiver
Sender
Publisher
Subscriber
Subscriber
Publisher
Architecture
Media
IPC Log Buffer
Media (UDP, InfiniBand, PCI-e 3.0)

36. Conductor
Sender
Receiver
Conductor
Receiver
Sender
Publisher
Subscriber
Subscriber
Publisher
Admin
Events
Architecture
Admin
EventsMedia
IPC Log Buffer
Media (UDP, InfiniBand, PCI-e 3.0)
Function/Method Call
Volatile Fields & Queues

37. ClientMedia DriverMedia Driver
Conductor
Sender
Receiver
Conductor
Receiver
Sender
Client
Publisher
Conductor Conductor
Subscriber
Subscriber
Publisher
Admin
Events
Architecture
Admin
EventsMedia
IPC Log Buffer
IPC Ring/Broadcast Buffer
Media (UDP, InfiniBand, PCI-e 3.0)
Function/Method Call
Volatile Fields & Queues

38. Is the Media Driver a broker?

39. Broker Relationship Status: Complicated…

40. Broker Relationship Status: Complicated…
1. Media Driver can be standalone

41. Broker Relationship Status: Complicated…
1. Media Driver can be standalone
2. Media Driver can be embedded

42. Broker Relationship Status: Complicated…
1. Media Driver can be standalone
2. Media Driver can be embedded
3. Core-to-Core memory handoff

43. Broker Relationship Status: Complicated…
1. Media Driver can be standalone
2. Media Driver can be embedded
3. Core-to-Core memory handoff
4. Isolation of protocol logic

44. Data Structures
• Maps
• IPC Ring Buffers
• IPC Broadcast Buffers
• ITC Queues
• Dynamic Arrays
• Log Buffers

45. Creates a
replicated persistent log
of messages
What Aeron does

46. Tail
File

47. Tail
File
Message 1
Header

48. Tail
File
Message 1
Header
Message 2
Header

49. Tail
File
Message 1
Header
Message 2
Header

50. Tail
File
Message 1
Header
Message 2
Header
Message 3

51. Tail
File
Message 1
Header
Message 2
Header
Message 3
Header

52. Persistent data structures can be
safe to read without locks

53. One big file that
goes on forever?

54. No!!!
Page faults, page cache churn,
VM pressure, ...

55. ActiveDirtyClean
Tail
Message
Header
Message
Header
Message
Header
Message
Header
Message
Header
Message
Header
Message
Header
Message
Header

56. How do we stay “wait-free”?

57. Tail
File
Message 1
Header
Message 2
Header
Message 3
Header
Message X
Message Y

58. Tail
File
Message 1
Header
Message 2
Header
Message 3
Header
Message X
Message Y

59. Tail
File
Message 1
Header
Message 2
Header
Message 3
Header
Message X
Message Y

60. Tail
File
Message 1
Header
Message 2
Header
Message 3
Header
Message X
Message Y
Header

61. Tail
File
Message 1
Header
Message 2
Header
Message 3
Header
Message X
Message Y
Header
Padding

62. Tail
File
Message 1
Header
Message 2
Header
Message 3
Header
Message Y
Header
Padding
File
Message X

63. Tail
File
Message 1
Header
Message 2
Header
Message 3
Header
Message X
Message Y
Header
Padding
File
Header

64. What’s in a header?

65. Data Message Header
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Version |B|E| Flags | Type |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-------------------------------+
|R| Frame Length |
+-+-------------------------------------------------------------+
|R| Term Offset |
+-+-------------------------------------------------------------+
| Session ID |
+---------------------------------------------------------------+
| Stream ID |
+---------------------------------------------------------------+
| Term ID |
+---------------------------------------------------------------+
| Encoded Message ...
... |
+---------------------------------------------------------------+

66. Header Evolution
1. Same header on wire & in memory
2. Frame Length originally 16-bits
3. IPv6-style Header Chains? Pfft!
4. Frame Alignment & Padding
5. Fragmentation? 2-bits
6. Position! Position! Position!

67. What is a position?

68. Unique identification of a byte
within each stream across time
(streamId, sessionId,
termId, termOffset)

69. count = currentTerm - initialTerm
(count * termLength) + termOffset
// termLength power of 2
bts = ntz(termLength)
(count << bts) + termOffset
Position

70. How do we replicate a log?

71. We need a Protocol of
messages

72. Sender Receiver

73. Receiver
Setup
Sender

74. Sender
Status
Receiver

75. DataData
Sender Receiver

76. DataData
Status
Sender Receiver

77. DataDataHeartbeat
Sender Receiver

78. DataData
NAK
Sender Receiver

79. Data
Sender Receiver

80. Protocol Evolution
1. 0-length Data Frames (SETUP & HB)
2. Ranged NAKs vs NAKing a Range
3. Send Padding & Frame Alignment!
4. Eliminating Special Cases

81. How are message streams
reassembled?

82. High Water Mark
File
Completed

83. High Water Mark
File
Message 1
Header
Completed

84. High Water Mark
File
Message 1
Header
Message 3
Header
Completed

85. File
Message 1
Header
Message 2
Header
Message 3
Header
Completed High Water Mark

86. How do we know what is
consumed?

87. Publishers, Senders,
Receivers, and Subscribers
all keep position counters

88. Counters are the key to
flow control and monitoring

89. Flow Control

90. Three Flow Control Windows
1. Publisher to Sender – Counter
2. Sender to Receiver – Status Messages
3. Receiver to Subscriber – Counter

91. Status Messages
Completed Position of Subscriber
+
Receiver Window

92. Status Message Generation
1. Term Rollover
2. Every ¼ Term Progression
3. On Timeout
Clocked by Sender Data Rate

93. Flow control strategies handle
composition of status from
multiple receivers

94. Safety
No Status Messages,
No Completed Progression,
No Sending

95. Congestion Control

96. Status Messages
Completed Position of Subscriber
+
Receiver Window

97. Dynamically adjust
Receiver Window based on loss

98. Loss, throughput, and buffer size
are all strongly related!!!

99. Pro Tip:
Know your OS network
parameters and how
to tune them

100. 4. What else did we
discover?

101. Some parts of Java really suck!

102. Some parts of Java really suck!
Unsigned Types?

103. Some parts of Java really suck!
Unsigned Types?
NIO (most of) - Locks

104. Some parts of Java really suck!
Unsigned Types?
NIO (most of) - Locks
Off-heap, PAUSE, Signals, etc.

105. Some parts of Java really suck!
Unsigned Types?
String Encoding
NIO (most of) - Locks
Off-heap, PAUSE, Signals, etc.

106. Some parts of Java really suck!
Unsigned Types?
String Encoding
NIO (most of) - Locks
Off-heap, PAUSE, Signals, etc.
Managing External Resources

107. Some parts of Java really suck!
Unsigned Types?
Off-heap, PAUSE, Signals, etc.
Selectors - GC
String Encoding
NIO (most of) - Locks
Managing External Resources

108. Some parts of Java are really nice!

109. Some parts of Java are really nice!
Tooling – IDEs, Gradle, HdrHistogram

110. Some parts of Java are really nice!
Tooling – IDEs, Gradle, HdrHistogram
Profiling – Flight Recorder

111. Some parts of Java are really nice!
Tooling – IDEs, Gradle, HdrHistogram
Bytecode Instrumentation
Profiling – Flight Recorder

112. Some parts of Java are really nice!
Tooling – IDEs, Gradle, HdrHistogram
Bytecode Instrumentation
Unsafe!!! + Java 8
Profiling – Flight Recorder

113. Some parts of Java are really nice!
Tooling – IDEs, Gradle, HdrHistogram
Bytecode Instrumentation
Profiling – Flight Recorder
The Optimiser
Unsafe!!! + Java 8

114. Some parts of Java are really nice!
Tooling – IDEs, Gradle, HdrHistogram
Bytecode Instrumentation
Profiling – Flight Recorder
The Optimiser – Love/Hate
Unsafe!!! + Java 8

115. Some parts of Java are really nice!
Tooling – IDEs, Gradle, HdrHistogram
Bytecode Instrumentation
Garbage Collection!!!
Profiling – Flight Recorder
Unsafe!!! + Java 8
The Optimiser – Love/Hate

116. 5. What’s Next?

117. Finished a few passes of
Profiling and Tuning

118. 20+ Million 40 byte
messages per second!!!

120. Replication
Services
Aeron Core
Persistence
Queueing
Performance
C/C++ Port
Batch Send/Recv
IPC
Infiniband
Multi Unicast Send
Stream Query

121. In closing…

123. Aeron: https://github.com/real-logic/Aeron
Twitter: @toddlmontgomery
Thank You!
Questions?